Features of Crawlomatic Scraper Plugin

v2.5.5 Update

• Auto-update scraped content: Automatically update scraped posts/pages/products if source site content changes.
• Unpublish if URL not available: Set posts/pages/products as drafts if source URL becomes unavailable (optional).

v2.5.1 Update

• Scrape WooCommerce variants: Scrape product variants from WooCommerce or Shopify stores.

v2.5.0 Update

• Scrape search engine results: Scrape custom keyword searches from Google or Bing.

v2.4.1 Update

• Scrape product image galleries: Scrape image galleries for WooCommerce products (non-product posts get post attachments).

v2.3.5 Update

• Execute custom JavaScript: Use JavaScript on scraped HTML when using headless browsers (Puppeteer, Tor, PhantomJS, or HeadlessBrowserAPI).

v2.2.1 Update

• RSS feed crawling: Crawl RSS feeds and scrape articles from linked content.

v2.2.0 Update

• Scrape JavaScript content: Scrape JavaScript-generated HTML from websites with HeadlessBrowserAPI.

v2.1.0 Update

• Scrape .onion websites (Dark Web): Use Tor Browser and Puppeteer for scraping .onion websites.

v2.0.0 Update

• Live Scraper shortcode: Use shortcode [crawlomatic-scraper] for more control over scraping.

v1.7.1 Update

• Sitemap crawling: Support for crawling sitemaps (with tutorial).

v1.6.5 Update

• Visual content selector: Ability to select visual content to scrape.

v1.6.0 Update

• Screenshots of crawled pages: Capture and use screenshots of crawled pages in generated posts.

v1.5.2 Update

• Shorten outgoing links: Automatically shorten links using the Shorte.st service for monetization.

v1.4.8 Update

• JavaScript execution: Support for running JavaScript on crawled pages (requires PhantomJS).

v1.4.4 Update

• Multiple proxies: Use multiple proxies for crawling, selected randomly for each page.

v1.4.0 Update

• Paginated crawling: Crawl articles across multiple pages.
• Import WooCommerce product prices: Automatically adjust prices for imported products (ideal for dropshipping).
• Increase imported product price: Adjust prices by a fixed value or multiply them.

v1.2.8 Update

• Paginated post importing: Import paginated posts into a single crawled post.

v1.2.4 Update

• Set proxies for crawling: Specify proxies for crawling pages.

v1.2.3 Update

• Crawl Google cache: Crawl pages from Google cache when direct crawling fails.

Additional Key Features

• Google Translate Support: Translate articles into selected languages.
• Text Spinner Support: Automatically spin text to generate unique content (supports services like The Best Spinner, SpinRewriter, and others).
• Customizable Post Status: Set generated post status (published, draft, pending, etc.).
• Shortcode for List of Posts: Display generated posts with a customizable shortcode.
• Respect Robots.txt: Option to respect the robots.txt files and meta tags of scraped websites.
• Product Prices: Import product prices for WooCommerce compatibility, with price modifications.
• Keyword Replacer Tool: Automatically replace keywords in posts with affiliate links.
• Scheduled Runs: Schedule automated scraping and content generation.
• Custom Field & Taxonomy Support: Support for adding custom fields and taxonomies to generated posts.
• Image Handling: Option to copy or link images locally.
• JSON Parsing: Parse JSON data using Regex for more complex scraping needs.
• Post Constraints: Define title and content length limits, and set conditions for posts to be generated based on content keywords.
• Free wordpress plugin.