Page 1 of 3 123 LastLast
Results 1 to 10 of 23
  1. #1
    Join Date
    Dec 2014
    Posts
    263

    Default How to Scrape Data That Is In a Span? Possible?

    Hi Thomas!

    I don't know if you know how to do it or not, but I was wondering if there is a technique needed to scrape data that is in a "Span".

    When I go to some sites where data is in tables, some of the info is oftentimes in a span together. The items can be selected, but no variable can be set for the items pulled from a span...I guess because it is still seen by the software as being part of the span.

    So, for example, a table of data might have 6 columns, and in the first column is a name, and the software can scrape that, but then in columns 2, 3, and 4, the data is all in a Span together, as far as the html is concerned. With some careful selection, data in only column 2 can be scraped, for example, but it can't be put in a variable, because it's still part of the span.

    Anyway, it seems the data in the span cannot be separated into individual columns, and variables assigned, because they are grouped as a span in the html. That's the gist of it.

    A bit of a puzzle. Lol.

    Thanks!

    Mel

  2. #2
    Join Date
    Dec 2008
    Posts
    3,228

    Default

    Quote Originally Posted by Mel Bel View Post
    Hi Thomas!

    I don't know if you know how to do it or not, but I was wondering if there is a technique needed to scrape data that is in a "Span".

    When I go to some sites where data is in tables, some of the info is oftentimes in a span together. The items can be selected, but no variable can be set for the items pulled from a span...I guess because it is still seen by the software as being part of the span.

    So, for example, a table of data might have 6 columns, and in the first column is a name, and the software can scrape that, but then in columns 2, 3, and 4, the data is all in a Span together, as far as the html is concerned. With some careful selection, data in only column 2 can be scraped, for example, but it can't be put in a variable, because it's still part of the span.

    Anyway, it seems the data in the span cannot be separated into individual columns, and variables assigned, because they are grouped as a span in the html. That's the gist of it.

    A bit of a puzzle. Lol.

    Thanks!

    Mel
    Hi Mel,

    I would need to see the page or html. There is always a way!


    Thomas

  3. #3
    Join Date
    Dec 2008
    Location
    Blue Planet
    Posts
    911

    Default

    Quote Originally Posted by Mel Bel View Post
    Hi Thomas!

    I don't know if you know how to do it or not, but I was wondering if there is a technique needed to scrape data that is in a "Span".

    When I go to some sites where data is in tables, some of the info is oftentimes in a span together. The items can be selected, but no variable can be set for the items pulled from a span...I guess because it is still seen by the software as being part of the span.

    So, for example, a table of data might have 6 columns, and in the first column is a name, and the software can scrape that, but then in columns 2, 3, and 4, the data is all in a Span together, as far as the html is concerned. With some careful selection, data in only column 2 can be scraped, for example, but it can't be put in a variable, because it's still part of the span.

    Anyway, it seems the data in the span cannot be separated into individual columns, and variables assigned, because they are grouped as a span in the html. That's the gist of it.

    A bit of a puzzle. Lol.

    Thanks!

    Mel


    Mel,
    care to PM me the url...ill take a shot at it for ya.It was easy on a WinAutomation and i am sure ESB Automation its doable easily...let me have a go and ill do a video for ya.
    Sorry...just saw this
    Robbie

  4. #4
    Join Date
    Dec 2014
    Posts
    263

    Default

    @Thomas and @Robbie,

    Warrior Plus affiliate offers page. I'm wondering how to scrape that data - sales, conversion rate, etc., are in the "span".

    https://warriorplus.com/affiliate/offers

    You'll have to login to W+ first to see the offers page.

    I can select the data in the span, but like I mentioned, that data cannot be set up with a variable output.

    Thanks!

    Mel

  5. #5
    Join Date
    Dec 2008
    Location
    Blue Planet
    Posts
    911

    Default

    Quote Originally Posted by Mel Bel View Post
    @Thomas and @Robbie,

    Warrior Plus affiliate offers page. I'm wondering how to scrape that data - sales, conversion rate, etc., are in the "span".

    https://warriorplus.com/affiliate/offers

    You'll have to login to W+ first to see the offers page.

    I can select the data in the span, but like I mentioned, that data cannot be set up with a variable output.

    Thanks!

    Mel


    I saw it. OK...will "try" to do a script tonight..else tomorrow morn. Will let u know how i go.
    Thomas may beet me to it..but let u know :P

  6. #6
    Join Date
    Dec 2014
    Posts
    263

    Default

    Thanks, Robbie. Well, Thomas has a lot on the table, so if you can figure it out that would be good and much appreciated. I tried a lot of different things to get it done so I wouldn't be bothering Thomas with it, but I finally had to give up and ask.

    I did look up how some people were accomplishing it with UIPath (scraping from a span of data), and they seem to be using some kind of workaround involving parent/child setups of some kind. I just briefly looked at it though because it was a long thread and seemed a bit complicated.

    Thanks again!

    Mel

  7. #7
    Join Date
    Dec 2008
    Location
    Blue Planet
    Posts
    911

    Default

    Hi Mel...
    The scrapping seems fine. Its just a little tricky to select them but it does a good job as you go through
    and "train" it to "reject" things it OVER selected...once you do that you fine tune it to get just the columns.
    You need to visually,every time, just scroll up/down with moue to confirm no other unwanted items are selected
    but you get a feel for it pretty quickly.
    No need to worry about the span.

    Here's some screen shots and i went and scrapped the first FIVE columns.


    1. http://prntscr.com/owz4rc <--first few

    2. http://prntscr.com/owz5mx <-- A few more.

    Its easy once you learn a few little nuances with the scraping.

    I will make a video tomorrow and make up a project file for you girls but its already 1AM and i
    better go to bed HAHA

    Seeya, all good....when u see me do it....it will just "click" for ya
    You'll be fine

    NIGHT


    Robbie
    Last edited by schmutly; 08-24-2019 at 07:02 PM.

  8. #8
    Join Date
    Dec 2008
    Posts
    3,228

    Default

    Thanks Robbie!


    Thomas

  9. #9
    Join Date
    Dec 2008
    Location
    Blue Planet
    Posts
    911

    Default

    Not an problem Thomas.

    I just got up not long ago, corrected some typos above but noticed that this
    is exactly what you need to watch out for.
    That is...you'll notice i made a stupid error.

    I selected the column names...literally and this will throw it OUT.
    You need to select the data and "write" the column names when you "add" the column
    to the one being scrapped.
    I see i was overly tired haha....it happens. So the PRICES are out-a-wack.

    Don't worry. I didnt save that example last night as i want to do the video & project for you where it
    will "ASK YOU" how many pages to scrape, then scrape that many.
    Then fill your database will that data AS WELL as grab an image (if available)...haven't checked that yet.


    My old script i did last year goes to Amazon(after you input a keyword you want to search) and scrapes the first page
    with the title,price,image,image of reviews...other stuff (going from memory) then puts it into a dashboard OVER TIME so
    you can get a feel for how its doing.
    I know i will need to do more videos....i am positive once you see "how" to do it from scratch it will click for you (and others)
    a lot faster. It's not that hard.
    Hardest part...to be totally honest? Its...um...like anything when you don't touch it for a while..muscle memory.
    I'm only just getting into that plugin (i hadn't looked at for 8+ months) and forgot a few things....until you "force" yourself
    into it...then the more you do it 'consistently (the key to it) you'll learn it fast and it comes quickly.

    Easy to say for spring chickens like you two girls LOL but us older types...need super cap, EverReady nuclear batteries to get us
    going. HAHA.

    OK...waffle over.
    Mel....i will 'try' to get this done today. I have a big distraction i forgot about. Sisters best friend is coming up as it's her birthday and
    were working on fixing her caravan...i need to do the re-wiring for her and help with some painting...so...if not ready by today then
    definitely sometime on Monday as i am HOME, yipeee

    Seeya, take care.
    (take care? where to take care....to the park to play? HAHA)
    Byeeee :P

    Robbie

  10. #10
    Join Date
    Dec 2014
    Posts
    263

    Default

    Thanks, Robbie. I've been able to scrape the data like you have shown, but where the problem has been is after getting the data, I can't assign any of the scraped data from the span to variables for output to the corresponding object properties.

    I'll get in my project file going today sometime and take some screenshots. The software basically won't recognize the data from the span being scraped so I can output the data in variables.

    Tc and thanks again!

    Mel

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may edit your posts
  •