Google maps scraping – important data and helpful tools

Even if you had a bad mark in Geography during your school days, you have used Google maps at least once. We will try to figure out how and when you may need Google maps data parsing.

Useful data for data crawling on Google maps

• The main reason to use Google maps scraper tool is the intention to get the list of organizations in the selected region. With the official Google Places API the following data can be parsed for each company:
• Name,
• Address,
• Phone number,
• Official site,
• Category,
• Customers Rate,
• Customers reviews,
• Google maps link.

Here is a part of JSON response:

Photos of the place can be received with the additional request by specifying the photos references:

2. The other sort of useful data on Google maps are geographic coordinates in bundle with addresses. Although Places API provides this information for the selected organization, you can use the special section of Geocoding API for converting address into coordinates or vice versa.

3. Google has several API sections for all kinds of routes data extraction. These sections methods are made more for real time using than parsing, but web scraping of some routes data really makes sense. For example Distance Matrix API provides the information about distances on the map.

How to use the data obtained from Google maps

It is hard to separate a few regular requests to Google maps API from Google maps scraping.

In first case you integrate Google maps into your service and always get up to date information from the source on demand. And this is preferable, because Google maps keeps the data about millions of places and routes and you can hardly parse it all.

In second case you use or write your own web data scraping application for Google maps to make a cut of data and work with it without connecting to Google maps. Sometimes if makes sense and we will talk a little bit when and why.

1. The most obvious way to use Google maps data is to collect phone numbers, addresses and websites of companies in your area:

• To form a potential customer base for your business,
• To find the firms selling the same product or service as you and analyze your business competitors,
• Or to find where to buy the product or service and have a chance to select from many options of different sellers.

2. You may need a great amount of data from Google maps if you want to create your own catalog of organizations. It should not only copy all the information directly from Google maps, it should be better and more convenient than the source to make people give preferences to your site – e.g., draw popular place times histogram or show the dynamics of average marks. That end you can combine Google places scraper with some other organizations catalogs scrapers to achieve completeness of data or use Google map data extraction tool regardless of API.

3. Delivery or taxi services that often apply to Google maps API can partly scrape Google maps to reduce the total amount of requests. Imagine that you deliver cargos to the long distances and have a couple of dozens of popular routes between cities. Web scraping google maps to form the list of your routes with exact distances once and for all would be a good idea.

4. Scientific or engineering work can require the help of Google map data scraper for different purposes. For example, someone is working with satellites and needs geolocation of different places, or someone is making climatic studies and wants to calculate the global distances on Earth to get cyclone velocity.

Web scraping Google maps: what tools are used to collect data

To write your own Google maps scraper Python is the programming language you may use optimally. First read the official Google Places API and get your scraper API key to make requests with. Then use requests module to search places passing the query parameters to the URL string:

Detailed information and examples of parameters are described in the relevant section of documentation.

Similarly send the HTTP request for data related to each place:

You will get JSON or XML response like we discussed above talking about Google maps data.

To scrape Google maps Python is not the only decision; it is possible to use PHP, JavaScript, Ruby or any other programming language depending on your current technology stack and tasks. And of course there are ready to use tools to scrape Google maps data without any programming skills.

One of these tools is Octoparse – general purpose web scrapper with the special template for parsing Google maps data. It takes only a few minutes to set up Octoparse crawler by giving it URLs or keywords as an input data and start extraction. Octoparse has built in RegEx tool, it supports IP changes and can run both on local machine and in a Cloud.

Google Chrome extensions like Web Scraper or Extrasaur Maps are able to help you with Google maps scraping too. Web Scraper solves a lot of parsing tasks and works with thousands of sites, but as Octoparse it can be set up for working with Google maps. Extrasaur Maps specializes only on maps scrapping, therefore it requires less settings. It works with the browser instance and doesn’t depend on Google maps API limitations. The output data Extrasaur Maps gives you in Excel format.

Google maps service is an open source of information, so feel free to select Google places scraper satisfying your business or scientific tasks, but try not to bother your local companies too much with your calls.