Scrape multiple webpages with changing URL with Python
I am scraping a website with a URL structure like this:
www.website.com/data?page=1
I would like to write a program that uses iteration to scrape data off all
of the pages, which start at 1 and end at various numbers, depending on
the data fields I select on the form.
I thought I could cut up the URL and use an iterator to increase that
page, but I can't concatenate a str and int object. Any advice?
No comments:
Post a Comment