tFileList is the frequent use component which allows you to list down files, from provided directory.
This post we will see different use case examples of tFileList.
tFileList has following global variable.
Global Variables List
this variable holds current file name without complete path.
this variables holds current file name with complete path.
this variable holds current file`s extension, if file has no extension then it will return null
this variable holds only directory name with path exclude file name.
this variables holds the number of files listed by tFilelist.
All the variables can be seen in OutLine window see the image.
tFileList Global variables list
Case 1: Iterate through all the files from directory and list down file details.
Create a job and add tFileList component then configure it as follows.
- tFileList configuration
- Directory= Directory path which you has files & folders.
- FileList Type= “Both” (Directory & files)
- Includes Sub Directories= “Yes”
- Case Sensitive= “No”
- Use Global… =”Yes”
- File Mask=”XYZ_*”
- Order By=”By Default”
- Order Action=”ASC”
- Add tFileProperties component next to tFileList and configured as follows.
- schema= default( it has read only schema)
- File = ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) (file name with path)
- Add tFileOutputDelimited component and configure as follows.
- File Name=”output path/FileList.txt”
- Row Separator=”\n”
- Field Separator=”|”
- Include Header = “Yes”
- Synch the schema
Flow: tFileList will list down all the files and iterate over each file, then we are using global variable “File with Path” inside tFileProperties, to get the details of file, and at last we are using tFileOutputDelemited component to create file list in file.
tFileProperties component gives us following information about file.
- abs_path- this column holds the complete file name with path.
- dir_name- this column holds only directory name.
- basename- this column holds only file Name.
- mode_string- this column give information about file permission whether file read only or has read write permission e.g. rw.
- size- file size in bytes.
- mtime- file creation/modification time in Long format.
- mtime_string- file creationg/modifcation datetime in string format. e.g. Tue Jun 09 13:43:32 IST 2015
Once you execute this it will list down all the files from directory with above information which can be utilise for further processing.
complete Job Design.
Case 2: Get latest file from directory.
Case 1 we have listed all the files & folder to the file, now we will get the latest file from directory.
create a job and add and configured component as follows.
Add tFilelist as stated in above case 1 and only change following properties.
- tFileList setting (Modifications)
- Order by= “By Modified Date”
by doing this modification tFilelist will list down all the files by their modified date and then we are using “order action” to Ascending.
tFileList will list all the files and sort it by modified date, by ascending order means newly created file placed in last to the list.
We need to iterate over these files and store the last created file into global variable for that we will use tSetGlobalVar component.
- Add tSetGlobalVar component next to tFilelist and set the following properties.
- Add new row to variables section using plus sign.
- In Key column add key=”FilePath”
- value = ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
- See the image below for more details.
- Add tFileProperties component below the tFileList and connect with OnSubJobOk and configure as follows.
- File= ((String)globalMap.get("FilePath"))
globalMap.get(“FilePath”) has our latest file path, which can retrieve using our key ( we set the key in tSetGlobalVar component) “FilePath”, it will return us object type so are casting it to String.
- Add tLogRow after tFileProperties and connect with main then select mode=table.
By executing this job will get the below output refer image for more details.
As a result we are getting our latest file, but you there can be situation wherein you need to parse file name to determine file created date and then need to process then do see How to get latest file from directory post.
By using “Order Action=Desc” you can get oldest file from directory.
Refer this post to delete older files than specified date.
If you are in situation wherein you need to create file with job run date, as well the occurrence of run need to be place in file name refer this post.
This post is useful for excluding files from same directory which not required for processing. you can specify group patterns for exclusion.
By referring these post you can be master of File Listing and tFileList component.
If you have any unique case which is not listed here then please do send I would like demonstrate here.