Is it possible to scrape data in 2 clicks? Scraping can really be a headache. There are so many sites, extracting information from which is not an obvious task. Sometimes we want to have ready-made templates for scraping – “click and use” – without any settings or reading the documentation.
For example, the popular AliExpress website provides the ability to select products by many parameters. Each of the parameters of a product (for example, size or price) can have its own price and own description. This price and description are shown only when you click on a product option. As a result, to extract all the data from the page, the application must either “click” on all the options or we must think of something else. In this case it would be reasonable to extract from the source code of the JSON page a set of source data about the product, which already requires some skill.
But how to combine the simplicity of the interface and at the same time provide a truly professional tool for scraping? 🤔
In our opinion, the ideal scraping application is first and foremost a simple scraping application. But simplicity and data scraping are not very overlapping concepts, are they? Nevertheless, fast scraping with Data Excavator is possible, and today we will talk about it.
Simplicity or versatility?
In the world of scraping, applications can be divided into two large categories.
The first category is applications that are made specifically for a certain site. For example, scraper only for Amazon.com or scraper only for Yelp.com. These applications are not suitable for scraping other sites, but they are surprisingly simple. They usually consist of a text input box and an “extract data” button.
The second category is versatile applications for scraping. These are products that have many settings and features and are suitable for extracting information from virtually any source. Such applications include Data Excavator. Yes, such applications are more complex both in understanding and in application. On the other hand, you can use them for almost any site.
By choosing between these categories, you will eventually understand which type of application is more convenient for you. You may not need to understand the features of complex applications, and there will be enough ready-made solution for one particular site. On the other hand, if the target site is not known to a wide audience (e.g., a cartographic site), you will somehow need something versatile.
How did we solve the problem of scraping “in 2 clicks”?
As we mentioned above, our application is a complex data scraper. It is suitable for almost any site. At the same time, it supports login and password authentication, custom JS scripts and much more. And the same functionality makes it complex. And in some cases it is very complex.
We always wanted to create a software product that would be understandable to a wide range of customers. We came up with a solution that would make data scraping for most sites simple and convenient. And this solution is a ready-made scraping templates.
Ready-made scraping templates
Inside our application there are two principal sections. The first section is called “my projects”. The second section is “ready-made scraping templates”. In the first section, you will find cards of working projects and management of working projects. The second section contains templates for scraping popular websites. Such sites as Amazon.com, Aliexpress.com, Yelp.com, BestBuy.com and many others.
The ready-made templates are created by our specialists and designed for professional data extraction. 🤖
In order to use a template, all you need to do is click on the “Create project” button. This will lead to the creation of a new project. Then click the “Save project” and “Run project” buttons. As a result, the application will extract data from the site, and you will not need any additional settings.
In the ready-made scraping templates tab we tried to take into account all the features of scraping the selected sites. If some template is intended for scraping E-commerce site, then in the scraping results you will get almost all data fields about the product (name, price, description, article, reviews, etc.), as well as a set of pictures. If some template is designed for scraping a site with ads (eg – Yelp), the results you get the most complete set of information about organizations from the site.
How to start using ready-made scraping templates?
Well, first of all you will need to download our application. You are guaranteed to get one month of free use when you activate the demo key. Next – go to the tab “Ready-made scraping templates” and find the right site for you. The library of ready-made templates is constantly growing, and with each month of ready-made settings becomes more and more.
Click the Create Project button, then save it and launch it. If the application has any clarifying questions, it will ask you them. Have a good scraping!