About dwetl

23 comments on “tHTMLInput- Parse HTML in Talend

  1. Hello there.

    Thanks a lot for this tutorial!!

    But… I don’t know hot to extract data from a website table.

    I don’t know how to search into source code, what can I use in Mapping area (Selector Code)?

    1. Hi Juan,
      Without selector it is very difficult to fetch the required details. if you dot`t know particular selector then you can directly use html tags like “p”, “span”, “div” or any other tag which you think it contains required information.

      Other wise you can use contact us page to send me secret message with website url and details which you want to extract.

      Thank You
      Dwetl team

      1. Thx you dwelt for this great component.
        It works for me. I have forgot to set the type to Object so I dont get the whole tag.

        An other question.
        How I can get the redirected links?
        Can you make a tutorial?

  2. Thank for your work ! Just a come back, I have tried some use cases and meet issue in those cases :
    – working behind proxy. I have used TSetProxy but connection doesn’t success
    – working with local file. I have used “file:///…” but connection doesn’t success. Reading Soup API, they talk about Parse method : “There is a sister method parse(File in, String charsetName) which uses the file’s location as the baseUri. This is useful if you are working on a filesystem-local site and the relative links it points to are also on the filesystem.”
    Implement the local file case will be really interesting ;))
    Regards

      1. So, can we parse a local file ?
        I get a “java.lang.NullPointerException” when using “file///c:/file.html” or “c:/file.html” for URL, is there another method to do it ?
        I am using the version 2 from Talend Exchange.
        Thanks for your work.

  3. how to call class value..?
    example:

    Hello World

    Output:democlass

    In the above code i need value of class..How can i write code for calling the value of class in tHTMLINPUT..?

    1. you can write a class in routines and then call the method in tHTMLINPUT component, if you put some example then we can suggest you better.

  4. Hi,
    I am trying to crawl HTML webpage table data’s using tHTMLInput component with particular tag attributes. Here If empty row data availed into table, Then Its repeating all column data again into that empty row. Below Job sample I am trying.

    tHTMLInput > tMap > tFileOutputDelimited

    Exact Question : How can I avoid empty tag attribute values after crawled data?

  5. Hi,
    I am trying to crawl HTML webpage table data’s using tHTMLInput component with particular tag attributes. Here If empty row data availed into table, Then Its repeating all column data again into that empty row. Below Job sample I am trying.
    tHTMLInput > tMap > tFileOutputDelimited
    Exact Question : How can I avoid empty tag attribute values after crawled data?

    1. Can you put more details like your source data and then output you get, i want to reproduce it, i can assist better.

  6. Hi there,

    first of all: thanks for a great component. I managed to scrape a table that import.io had no way of scraping. However, as some people before me, I struggle with extracting ID’s, links or other ‘embedded’ information.

    The most straightforward example is regarding the ID. I want to extract the ID-value of every row. At the moment I use “[id]” as selector code with the column type set to “Object”. This returns the whole line as html. Afterwards I clean it up in tMap, but it’s not very easy this way.

    I also recognise the issues someone had with an ‘empty row’. This occurs when in some records there is a field (in my case a with “[class*=lineuprow]”) and in others there is not. If this is not available, the tHTMLInput just returns all values it can find in the ‘Parent Element’. My workaroud for this is to extract the first field of every record and use tMap to suppress all records of “[class*=lineuprow]” that start with that same value. But I would prefer tHTMLInput to return NULL in case when it cannot find the Selector Code.

    If you could help me with the first issue, this would be greatly appreciated. The second is a nice to have.

    Kind regards,

    Pepe

Leave a Reply to Rodrigo Cesar Antunes Cancel reply

Your email address will not be published. Required fields are marked *