Pulling Twitter Updates Using Talend Open Studio -Part II

Pulling Twitter Updates Using Talend Open Studio -Part II
3.33 (66.67%) 6 votes

In previous post you have seen how to get a user ID`s from twitter API. In this post we will see how can we get the Users details.

Requirement for Demo. 
Talend Open Studio. 
Twitter API Access. 
JDK installed. 

Access to below API.

followers/ids
users/show
statuses/friends
statuses/followers

To get user`s detail we will have to use users/show API so make sure you are able to access this API. Once you  click on mention API you will gate a XML file which has many details, so making it simple and understandable we will take below details from XML, if you want then you can take all the details. 

Create a Job in Talend Open Studio and follow the step for creating mapping/Schema of users/show XML.


Click on Metadata Node, and right Click on File XML node, then click on Create File XML option from Pop up.

Provide mapping name and users/show XML File. 

Once done go to the next Tab and configure all the properties like below screen. and select below listed fields for display.



Our Xpath Loop expression is: /user/status
select below list of field using Ctrl+click and drag and drop to “Fileds to extract” and click on “Refresh Preview” button to make sure you have parsed XML properly. 

created_at
description
es_count
favourites_count
followers_count
following
friends_count
id
location
name
screen_name 
time_zone
usl
verified

Now we have ready our sample file with twitter user details. we have to store this information into CSV file, hence you need to drag and drop tFileOutPutDelimited, Drag and Drop schema mapping we created recently for XML on designer and select tFileInputXML. Connect tFileInputXML to tFileOutPutDelimited using Main connector , and synch source schema to tFileOutPutDelimited component, 

Give the output file path and name other configuration. once done execute the job to sure every thing is working fine. your final job look like below with output. 




Output



Here we have completed two part of Twitter API, one to get the user ID`s and other one is to get user details. 

In next part of this post we will integrate both jobs in single one to retrieve each user id and their details in CSV file. 




About Umesh

I am Software consultant with approx 7 years of experience mainly in Business Intelligence and data warehousing assignments using Talend. Writing is not my passion but i am doing it to help others. if you have any special case where you want me to demonstrate then please post me.

Leave a Reply

Your email address will not be published. Required fields are marked *