Skip to content

scrap ebay market-place for completed ps5 & xbox-series-x sales

Notifications You must be signed in to change notification settings

monyedavid/ebay-scrape

Repository files navigation


Logo

Ebay Autonomous Scrapping

A puppeteer bot to collect completed sales information for the playstation 5 & xbox series x next gen.consoles
Explore the API docs »

View Demo (GQL playground) · Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Getting Started
  3. Usage

About The Project

Product Name Screen Shot

A simple bot to extract(every 45 mins) and store the latest completed sales infomration on the next-gen consoles playstation 5 & xbox series-x run in scalable containers

Built With

Getting Started

To set up project locally, please note that a demo project has been hosted on here

Prerequisites

Installation

  1. Clone the repo
    git clone https://github.com/rasheeddavid/ebay-scrape
  2. Cd into root directory
    cd ebay-scrape
  3. Run docker-compose
    docker-compose up -d

Usage (Queries)

get

Retrieve paginated data collected by bot filtered by console & date, sorted by most recent

get_demo

daily

Retrieve paginated completed sales for the current date, updated every 45mins

demo_2

get - from source

Retrieve paginated data(25/page) directly from source, sorted by most recent

demo_2

real time updates - from updates

Retrieve data from source in real time (scheduled to run every 10 minutes to avoid a ban). Runs in websockets & graphql subscriptions

demo_2

demo_2

Project Structure

Where find stuff 🔍

directory layout

.
├── build                       # Compiled files (javascript source files)
├── docker                      # Dockerfile(s)
├── src                         # Source files
  ├── Database            # DB entity objects & migrations
  ├── GraphQL             # Resolvers & Schemas
  ├── Services
      ├── bot                # Handle data collection
      ├── cron               # Scheduling data collection
      ├── ebay.scrappr       # Handle Scrappping ebay
      └── scrapped.data      # Handle Queries (bot-DB-interation)
  ├── Utils               # Utilities
  ├── apollo.server.ts    # Application entry
  └── index.ts
├── ormconfig.json              # Type-Orm configurations
└── README.md