tFileList use case example iterates on files or folders

tFileList use case example iterates on files or folders
5 (100%) 10 votes

tFileList retrieves a set of files or folders based on a filemask pattern and iterates on each unity.

tFileList is the frequent use component which allows you to list down files, from provided directory.

This post we will see different use case examples of tFileList.

tFileList has following global variable.

Global Variables List

  • ((String)globalMap.get("tFileList_1_CURRENT_FILE"))

this variable holds current file name without complete path.

  • ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

this variables holds current file name with complete path.

  • ((String)globalMap.get("tFileList_1_CURRENT_FILEEXTENSION"))

this variable holds current file`s extension, if file has no extension then it will return null

  • ((String)globalMap.get("tFileList_1_CURRENT_FILEDIRECTORY"))

this variable holds only directory name with path exclude file name.

  • ((Integer)globalMap.get("tFileList_1_NB_FILE"))

this variables holds the number of files listed by tFilelist.

All the variables can be seen in OutLine window see the image.

tFileList Global variables list

tFileList Global variables list

Case 1: Iterate through all the files from directory and list down file details.

Create a job and add tFileList component then configure it as follows.

  • tFileList configuration
    • Directory= Directory path which you has files & folders.
    • FileList Type= “Both” (Directory & files)
    • Includes Sub Directories= “Yes”
    • Case Sensitive= “No”
    • Use Global… =”Yes”
    • File Mask=”XYZ_*”
    • Order By=”By Default”
    • Order Action=”ASC”
  • Add tFileProperties component next to tFileList and configured as follows.
    • schema= default( it has read only schema)
    • File((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) (file name with path)
  • Add tFileOutputDelimited component and configure as follows.
    • File Name=”output path/FileList.txt”
    • Row Separator=”\n”
    • Field Separator=”|”
    • Append=”Yes”
    • Include Header = “Yes”
    • Synch the schema

Flow: tFileList will list down all the files and iterate over each file, then we are using global variable “File with Path” inside tFileProperties, to get the details of file, and at last we are using tFileOutputDelemited component to create file list in file.

tFileProperties component gives us following information about file.

  •  abs_path- this column holds the complete file name with path.
  • dir_name- this column holds only directory name.
  • basename- this column holds only file Name.
  • mode_string- this column give information about file permission whether file read only or has read write permission e.g. rw.
  • size- file size in bytes.
  • mtime- file creation/modification time in Long format.
  • mtime_string- file creationg/modifcation datetime in string format. e.g.  Tue Jun 09 13:43:32 IST 2015 

Once you execute this it will list down all the files from directory with above information which can be utilise for further processing.

complete Job Design.

File Properties in debug mode
File Properties in debug mode

Case 2: Get latest file from directory.

Case 1 we have listed all the files & folder to the file, now we will get the latest file from directory.

create a job and add and configured component as follows.

Add tFilelist as stated in above case 1 and only change following properties.

  • tFileList setting (Modifications)
    • Order by= “By Modified Date”

by doing this modification tFilelist will list down all the files by their modified date and then we are using “order action” to Ascending.

tFileList will list all the files and sort it by modified date, by ascending order means newly created file placed in last to the list.

We need to iterate over these files and store the last created file into global variable for that we will use tSetGlobalVar component.

tSetGlobalVar Settings.

  • Add tSetGlobalVar component next to tFilelist and set the following properties.
    • Variables
      • Add new row to variables section using plus sign.
      • In Key column add key=”FilePath”
      • value = ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
      • See the image below for more details.
tSetGlobalVar Setting for Latest file
tSetGlobalVar Setting for Latest file
  • Add tFileProperties component below the tFileList and connect with OnSubJobOk and configure as follows.
    • File= ((String)globalMap.get("FilePath"))

globalMap.get(“FilePath”) has our latest file path, which can retrieve using our key ( we set the key in tSetGlobalVar component) “FilePath”, it will return us object type so are casting it to String.

  • Add tLogRow after tFileProperties and connect with main then select mode=table.

By executing this job will get the below output refer image for more details.

tFilelist- Latest file from directory
tFilelist- Latest file from directory

As a result we are getting our latest file, but you there can be situation wherein you need to parse file name to determine file created date and then need to process then do see  How to get latest file from directory post.

By using “Order Action=Desc” you can get oldest file from directory.

Case 3. Delete files more than n number of days old. 

Refer this post to delete older files than specified date.

Case 4. Create File with Incremental Number on each run of job.

If you are in situation wherein you need to create file with job run date, as well the occurrence of run need to be place in file name refer this post.

Case 5. tFileList Exclude Mask, to exclude unwanted files from listing. 

This post is useful for excluding files from same directory which not required for processing. you can specify group patterns for exclusion.

By referring these post you can be master of File Listing and tFileList component.

If you have any unique case which is not listed here then please do send I would like demonstrate here.

About dwetl

4 comments on “tFileList use case example iterates on files or folders

  1. I have task in which i have one delimited file(file1) with 2 text fields. not i have a folder in which is have list of files. i want that only file name to be added in my file1 that matches first field in file name. Iterate through the list of file keeping one file name as a mask at a time, get the file name and update file 1 – third field with that name. How can it be done

    1. Hi Tapan,

      Provide us more details like, source file name, file which contains the file name, and expected result, i will provide you complete job flow.

      Thanks
      Umesh

  2. Hi,

    Thanks for the information.

    In my scenario i want to forward the latest file into tFileInputExcel. I have applied case 2
    Please let me know what other changes are required?

    Thanks,
    Kuldeep

Leave a Reply to dwetl Cancel reply

Your email address will not be published. Required fields are marked *