Skip to content

Latest commit

 

History

History
272 lines (220 loc) · 26.5 KB

README.md

File metadata and controls

272 lines (220 loc) · 26.5 KB
Raw

Awesome Python Testing Awesome

Collection of awesome Python resources for testing and generating test data.

Contents

Assertions

  • believe - A python package for json/dictionary validation.
  • easycheck - A collection of assertion-like functions to be used in code, where assertion themselves should be avoided; easycheck includes also function aliases to be used in unit testing.
  • expects - Expects is an expressive and extensible TDD/BDD assertion library for Python.
  • expycted - Another Python expect pattern implementation. Simple, intuitive and approachable, with ability to plug in to any testing framework that relies on assertions.
  • dirty-equals - A python library that (mis)uses the __eq__ method to make python code (generally unit tests) more declarative and therefore easier to read and write.
  • Precisely - Write precise assertions so you only test the behaviour you're really interested in.
  • PyHamcrest - A framework for writing matcher objects, allowing you to declaratively define "match" rules.
  • pytest_cache_assert - Cache assertion data to simplify regression testing of complex serializable data.
  • sure - An idiomatic assertion toolkit with human-friendly failure messages, inspired by RSpec Expectations and should.js.

Behavior-driven Development

  • behave - is behavior-driven development, Python style.
  • lettuce - Behavior-driven-development tool for python, inspired by Cucumber for Ruby.
  • mamba - The definitive testing tool for Python. Born under the banner of BDD.

Code Coverage

Design by Contract

  • deal - Design by contract for Python with static checker and test generation.
  • icontract - Design-by-contract in Python3 with informative violation messages and inheritance.
  • pact-python - Python version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.

Fake Data

  • autofaker - designed to minimize the setup/arrange phase of your unit tests by removing the need to manually write code to create anonymous variables as part of a test cases setup/arrange phase.
  • faker - A Python package that generates fake data.
  • fake2db - Fake database generator.
  • genuine-fake - Genuine Fake means an imitation of a (usually) valuable object that is so good that it is, to all intents and purposes, identical.
  • mimesis - A Python library that helps you generate fake data.
  • radar - Generate random datetime / time.

Load Testing

  • Dynamic Workload Model - Code to generate dynamic workload model. Useful for testing autoscaling in cloud or mimicking different load profile for different scenario.
  • Grasshopper - A lightweight framework for performing load tests against an environment, primarily against an API. Grasshopper glues Locust, Pytest, some plugins (namely Locust InfluxDBListener ) and some custom code to provide a package that makes authoring load tests simple with very little boilerplate needed.
  • Grizzly - is a framework to be able to easily define load scenarios, and is mainly built on-top of Locust and Behave.
  • Locust - Scalable user load testing tool written in Python.
  • pynonymizer - is a universal tool for translating sensitive production database dumps into anonymized copies.

Memory Management

  • filprofiler - A Python memory profiler for data processing and scientific computing applications.
  • Guppy 3 - a Python programming environment & heap analysis toolset.
  • mem_top - shows top suspects for memory leaks in your Python program.
  • Pympler - is a development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application.
  • tracemalloc - is a debug tool to trace memory blocks allocated by Python.

Mock and Stub

  • Aioresponses - is a helper for mock/fake web requests in python aiohttp package.
  • Cornell - record & replay mock server.
  • doublex - Powerful test doubles framework for Python.
  • Flexmock - is a testing library for Python that makes it easy to create mocks, stubs and fakes.
  • freezegun - Travel through time by mocking the datetime module.
  • httmock - A mocking library for requests for Python 2.6+ and 3.2+.
  • httpretty - HTTP request mock tool for Python.
  • Kesha - A web service with a user interface for testing http requests and web hooks.
  • mock - (Python standard library) A mocking and patching library.
  • Mockafka - Python library designed for mocking Kafka in a testing environment. It simplifies testing Kafka-integrated applications by providing an in-memory mock for aiokafka and confluent-kafka-python.
  • mocket - A socket mock framework with gevent/asyncio/SSL support.
  • Mockintosh - aims to provide usual HTTP mock service functionality with small resource footprint, making it friendly for microservice applications.
  • moto - allows you to easily mock out tests based on AWS infrastructure.
  • Pretend - is a library to make stubbing with Python easier.
  • responses - A utility library for mocking out the requests Python library.
  • time-machine - Travel through time in your tests.
  • trustme - gives you a fake certificate authority (CA) that you can use to generate fake TLS certs to use in your tests.

Mutation Testing

  • Cosmic Ray - makes small changes to your source code, running your test suite for each one.
  • Mutatest - Python mutation testing.
  • Mutmut - is a mutation testing system for Python, with a strong focus on ease of use.
  • MutPy - MutPy is a mutation testing tool for Python 3.x source code
  • xmutant.py - Python 3.6 bytecode based mutation analysis framework

Object Factories

  • factory_boy - A test fixtures replacement for Python.
  • mixer - Another fixtures replacement. Supports Django, Flask, SQLAlchemy, Peewee and etc.
  • Model Bakery - offers you a smart way to create fixtures for testing in Django.
  • polyfactory - A simple and powerful mock data generation library, based around type hints and supporting dataclasses, typed-dicts, pydantic models, msgspec structs and more.

Penetration Testing

  • fsociety - A Modular Penetration Testing Framework.
  • python-pentest-tools - Python tools for penetration testers.
  • fencer - Fencer is an automated API security testing tool. It's an experimental project to see how much of the API security testing process can be automated.

Property Based Testing

  • Atheris - is a coverage-guided Python fuzzing engine. It supports fuzzing of Python code, but also native extensions written for CPython.
  • hypofuzz - Adaptive fuzzing of Hypothesis tests.
  • Hypothesis - is an advanced Quickcheck style property based testing library.
  • hypothesis-auto - An extensions for Hypothesis that provides fully automatic testing for type hinted functions.
  • hypothesis-crosshair - Add the power of solver-based symbolic execution to your Hypothesis tests with CrossHair.
  • icontract-hypothesis - Instead of writing manually the Hypothesis search strategies for a function, icontract-hypothesis infers them based on the function's precondition.

Rest API Testing

  • behave-restful - BDD Framework to Test REST Services and APIs.
  • cherrybomb - CLI tool that helps you avoid undefined user behaviour by auditing your API specifications, validating them and running API security tests.
  • Dredd - is a language-agnostic command-line tool for validating API description document against backend implementation of the API.
  • HttpRunner - is a simple & elegant, yet powerful HTTP(S) testing framework.
  • hypothesis-graphql - Hypothesis strategies for GraphQL schemas, queries and data.
  • Schemathesis - A tool for automatic property-based testing of web applications built with Open API / Swagger specifications.
  • playback - A Python decorator-based framework that lets you "record" and "replay" operations (e.g. API requests, workers consuming jobs from queues).
  • RESTler - is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
  • Tavern - is a pytest plugin, command-line tool and Python library for automated testing of APIs, with a simple, concise and flexible YAML-based syntax.
  • Zato API Test - API testing in pure English. No programming needed. Implemented and extendable in Python.

Snapshot Tests

  • ApprovalTests - work by comparing the test results to a golden master.
  • SnapshotTest - is a way to test your APIs without writing actual test cases.
  • syrupy - is a zero-dependency pytest snapshot plugin. It enables developers to write tests which assert immutability of computed results.
  • VCR.py - Record and replay HTTP interactions on your tests.

Speed

  • Partial Testing - Run only the tests that are relevant for your changes.
  • Pytest-incremental - analyses your project structure and file modifications between test-runs to modify the order tests are executed and de-select tests.
  • pytest-testmon - Selects tests affected by changed files. Continuous test runner when used with pytest-watch.
  • Awesome pytest speedup - A checklist of best practices to speed up your pytest suite.

Static Checks

  • awesome-flake8-extensions - A curated list of awesome flake8 extensions.
  • awesome-python-typing - Collection of awesome Python types, stubs, plugins, and tools to work with them.
  • Bandit - is a tool designed to find common security issues in Python code.
  • flake8 - is a python tool that glues together pep8, pyflakes, mccabe, and third-party plugins to check the style and quality of python code.
  • pyanalyze - A tool for programmatically detecting common mistakes in Python code, such as references to undefined variables and some categories of type mismatches.
  • pyflakes - A simple program which checks Python source files for errors.
  • Pylint - A Python static code analysis tool which looks for programming errors, helps enforcing a coding standard, sniffs for code smells and offers simple refactoring suggestions.
  • Refurb - A tool for refurbishing and modernizing Python codebases.
  • ruff - An extremely fast Python linter, written in Rust.
  • slotscheck - Find mistakes in your __slots__ definitions.
  • Typecheckers - A list of Python type checkers.
  • typos - A Source code spell checker that finds and corrects spelling mistakes in source code.

Test Runners

  • green - A clean, colorful test runner.
  • Nox - is a command-line tool that automates testing in multiple Python environments, similar to tox. Unlike tox, Nox uses a standard Python file for configuration.
  • tox - Auto builds and tests distributions in multiple Python versions.

Testing Frameworks

  • async-asgi-testclient - A framework-agnostic library for testing ASGI web applications.
  • awesome-pytest - A curated list of awesome pytest resources.
  • awesome-robotframework - A curated list of awesome Robot Framework resources and libraries.
  • doctest - (Python standard library) The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.
  • hammett - Fast python test runner, compatible with a subset of pytest.
  • maelstrom - Maelstrom is a fast Rust and Python test runner that runs every test in its own container. Tests are either run locally or distributed to a clustered job runner.
  • nose2 - The successor to nose, based on unittest2.
  • perftester - A lightweight framework for performance testing of Python functions; allows for testing of performance in terms of execution time and memory usage.
  • promptimize - a prompt engineering evaluation and testing toolkit. It accelerates and provides structure around prompt engineering at scale with confidence, bringing some of the ideas behind test-driven development (TDD) to engineering prompts.
  • pytest - A mature full-featured Python testing tool.
  • sundew - Sundew is a testing framework for Python, implementing a new approach to testing. One that combines functional programming concepts and the general best practices for writing tests that we already know, and enforces them in a way that enables some really powerful features that make testing easier, enjoyable, and more effective.
  • Robot Framework - A generic test automation framework.
  • testbook - A unit testing framework extension for testing code in Jupyter Notebooks.
  • unittest - (Python standard library) Unit testing framework.
  • vedro - Pragmatic testing framework for Python
  • Ward - is a modern test framework for Python with a focus on productivity and readability.
  • xdoctest - A rewrite of Python's builtin doctest module (with pytest plugin integration) with AST instead of REGEX.

Tools

  • Auger - Project to automatically generate unit tests for Python code.
  • CrossHair - An analysis tool for Python that blurs the line between testing and type systems.
  • ghostwriter - Writing tests with Hypothesis frees you from the tedium of deciding on and writing out specific inputs to test.
  • importlab - A library that automatically infers dependencies for Python files. Importlab's main use case is to work with static analysis tools that process one file at a time, ensuring that a file's dependencies are analysed before it is.
  • Klara - Static analysis tools to automatic generate test case, based on SMT (z3) solver, with a powerful ast level inference system.
  • logot - Test whether your code is logging correctly.
  • Pifpaf - A suite of fixtures and a command-line tool that allows to start and stop daemons for a quick throw-away usage. This is typically useful when needing these daemons to run integration testing.
  • Pynguin - A tool that allows developers to generate unit tests automatically.
  • pytest-codegen - Pytest-codgen will statically analyze your code to create pytest function stubs.
  • pytestify - Automatically convert unittests to pytest.
  • pytest-mock-generator - A pytest fixture wrapper for mock_autogen.
  • teyit - A static analyzer and a refactoring tool to rewrite your unittest assertions in the right way.

UI Testing

  • Flybirds - is a front-end UI automation test framework based on BDD mode, providing a series of out-of-the-box tools and complete documentation.
  • Golem - is a test framework and a complete tool for browser automation. Tests can be written with code in Python, codeless using the web IDE, or both.
  • helium - is great for web automation. Helium makes it easier to use.
  • Lost Pixel - is an open source visual regression testing tool. Run visual regression tests on your Storybook and Ladle stories and on your application pages.
  • LuluTest - is an open source browser automation framework using Python and Selenium.
  • PyAutoGUI - is a cross-platform GUI automation Python module for human beings.
  • pytest-ui-automatic - Playwright Python tool practice pytest pytest-bdd screen-play page-object allure cucumber-report.
  • selene - User-oriented Web UI browser tests in Python (Selenide port).
  • Selenium - Python bindings for Selenium WebDriver.
  • SeleniumBase - is an all-in-one Python framework for automated browser testing. Tests are run with "pytest", and use WebDriver APIs for web-page interaction.
  • sixpack - A language-agnostic A/B Testing framework.
  • splinter - Open source tool for testing web applications.
  • squape - is a package that extends Squish API providing convenience tools for everyday automated test cases development.

Resources

Articles

Books

Videos

Related

  • Awesome Python - A curated list of awesome Python frameworks, libraries, software and resources.
  • Python test automation - A comprehensive curated list of python test automation frameworks, tools, libraries and software to help software engineers easily bootstrap test automation on python.
  • commit-check - Check commit message formatting, branch naming, commit author, email, and more.