Google Maps is one of the most requested data sources in my freelance work: business listings for lead generation, market mapping (for example music studios and art spaces in Dubai and Abu Dhabi), and customer reviews for competitor research. Here is what I've learned building these scrapers.
What clients usually want
- Listings: name, rating, number of reviews, phone, opening status, address, latitude/longitude and website.
- Reviews: date, star rating, review text and common keywords for one or many places.
Similar work extends to Google Play reviews, Trustpilot and marketplace reviews, with the same patterns.
1. The results list is an infinite scroll
Search results load in batches as you scroll the side panel, not the page. The scraper scrolls that specific panel, waits for new cards to appear and stops when the "end of list" marker shows up or no new cards arrive after a few attempts. Counting cards before and after each scroll is a simple and reliable stop condition.
2. Details load lazily
Phone numbers, websites and opening hours only appear after a place is opened. Each place is visited, and the scraper waits for specific elements instead of sleeping for a fixed time. That is faster and much less flaky.
3. Coordinates are in the URL
The latitude and longitude of a place are embedded in its URL. Parsing them from there is more reliable than hunting for them in the page.
4. Long runs need a persistent profile
For jobs with thousands of places I run Chrome with a persistent user-data profile. The session looks consistent, there are fewer interruptions, and a crashed run can resume.
5. Save as you go
Results are written to Excel in batches and de-duplicated by place URL. If anything fails at listing 3,000, the first 2,999 are already safe, and a restart skips them.
Doing it responsibly
I collect publicly visible business information, keep request rates modest and avoid personal data unless the project clearly needs it. For review analysis, reviewer names are usually unnecessary and can be dropped or anonymised.