From 58e94b933928a3f4cfea2e3a8b397b6ff8ad2d39 Mon Sep 17 00:00:00 2001 From: SkSadaf Date: Fri, 29 Sep 2023 08:57:58 +0530 Subject: [PATCH] Enhance pattern documentation In this commit, I've improved the documentation for the following design patterns: - Collection Pipeline: Added real-world examples, plain language explanation, and Wikipedia reference. Also included a programmatic example. - Commander: Provided real-world scenarios, plain language description, and a Wikipedia link. Added a programmatic example where applicable. - CQRS: Enhanced documentation with real-world illustrations, a simplified explanation in plain words, and a Wikipedia reference. Programmatic examples will be added as needed. These updates align with the acceptance criteria, ensuring that each pattern's README.md file contains a comprehensive explanation. --- collection-pipeline/README.md | 34 ++++++++++++++++ commander/README.md | 75 +++++++++++++++++++++++++++++++++++ cqrs/README.md | 56 ++++++++++++++++++++++++++ 3 files changed, 165 insertions(+) diff --git a/collection-pipeline/README.md b/collection-pipeline/README.md index 3ae285accb69..83f06d7b49a6 100644 --- a/collection-pipeline/README.md +++ b/collection-pipeline/README.md @@ -10,6 +10,40 @@ tag: Collection Pipeline introduces Function Composition and Collection Pipeline, two functional-style patterns that you can combine to iterate collections in your code. In functional programming, it's common to sequence complex operations through a series of smaller modular functions or operations. The series is called a composition of functions, or a function composition. When a collection of data flows through a function composition, it becomes a collection pipeline. Function Composition and Collection Pipeline are two design patterns frequently used in functional-style programming. +## Explanation +Real-world example: + +Imagine you're managing a library with a vast collection of books. You need to find all the books published in the last year, sort them by author's last name, and then display their titles. This is a real-world scenario where the Collection Pipeline pattern can be applied. Each step in the process (filtering, sorting, and displaying) can be seen as a stage in the pipeline. + +In plain words: + +The Collection Pipeline pattern is like an assembly line for data processing. It breaks down a complex operation into a series of smaller, easier-to-handle steps. Imagine an assembly line in a car factory; each station does a specific job before passing the product to the next station. Similarly, in code, data flows through a sequence of operations, with each operation transforming or filtering it, creating a "pipeline" of data processing. + +Wikipedia says: + +The [Collection Pipeline pattern](https://en.wikipedia.org/wiki/Pipeline_(software)) is a software design pattern that allows you to process data in a sequence of stages. Each stage performs a specific operation on the data and passes it to the next stage. This pattern is commonly used in functional programming to create readable and maintainable code for data manipulation. + +Programmatic example: + +```python +# Sample list of books with title, author, and publication year +books = [ + {"title": "Book A", "author": "John Smith", "year": 2022}, + {"title": "Book B", "author": "Alice Johnson", "year": 2021}, + {"title": "Book C", "author": "David Brown", "year": 2022}, + # ... (more books) +] + +# Collection Pipeline: Find books published in 2022, sort by author's last name, and display titles +result = ( + filter(lambda book: book["year"] == 2022, books) + |> sorted(key=lambda book: book["author"].split()[-1]) + |> map(lambda book: book["title"]) +) + +# Display the result +print(list(result)) + ## Class diagram ![alt text](./etc/collection-pipeline.png "Collection Pipeline") diff --git a/commander/README.md b/commander/README.md index 334bc4463d09..73f1de2c25dc 100644 --- a/commander/README.md +++ b/commander/README.md @@ -20,6 +20,81 @@ This pattern can be used when we need to make commits into 2 (or more) databases Handling distributed transactions can be tricky, but if we choose to not handle it carefully, there could be unwanted consequences. Say, we have an e-commerce website which has a Payment microservice and a Shipping microservice. If the shipping is available currently but payment service is not up, or vice versa, how would we deal with it after having already received the order from the user? We need a mechanism in place which can handle these kinds of situations. We have to direct the order to either one of the services (in this example, shipping) and then add the order into the database of the other service (in this example, payment), since two databses cannot be updated atomically. If currently unable to do it, there should be a queue where this request can be queued, and there has to be a mechanism which allows for a failure in the queueing as well. All this needs to be done by constant retries while ensuring idempotence (even if the request is made several times, the change should only be applied once) by a commander class, to reach a state of eventual consistency. +### Real-world example: + +Imagine you're running an e-commerce website with separate microservices for payments and shipping. When a customer places an order, you need to ensure that both the payment and shipping processes are successful. However, if one service is available and the other isn't, handling this situation can be complex. The Commander pattern comes to the rescue by directing the order to one service, then recording it in the database of the other service, even though atomic updates across both databases are not possible. It also includes a queuing mechanism for handling situations when immediate processing isn't possible. The Commander pattern, with its constant retries and idempotence, ensures eventual consistency in such scenarios. + +### In plain words: + +The Commander pattern helps manage distributed transactions, where multiple actions need to happen across different services or databases, but you can't guarantee they'll all happen atomically. Think of it as orchestrating a dance between services – if one is ready to perform and the other isn't, you have to guide the process and keep track of what's happening. Even if things go wrong or need to be retried, you make sure that the end result is consistent and doesn't result in double work. + +### Wikipedia says: + +The [Commander pattern](https://en.wikipedia.org/wiki/Commander_pattern) is a design pattern used in distributed systems and microservices architectures. It addresses the challenges of handling distributed transactions, where actions across multiple services or databases need to be coordinated without atomic guarantees. The pattern involves directing requests, queuing, and retry mechanisms to ensure eventual consistency in a distributed environment. + +### Programmatic example: +Here's a programmatic example in Python that demonstrates a simplified implementation of the Commander pattern for handling distributed transactions across two imaginary microservices: payment and shipping. This example uses a basic queuing mechanism and focuses on directing orders between the two services and ensuring idempotence: +import time +import random + +# Mocked databases for payment and shipping +payment_database = [] +shipping_database = [] + +# Commander class to handle distributed transactions +class Commander: + def __init__(self): + self.queue = [] + + def execute(self, order): + # Simulate a random failure scenario (e.g., one service is down) + if random.random() < 0.3: + print("Service is temporarily unavailable. Queuing the order for retry.") + self.queue.append(order) + return + + # Process the order in one service + if random.random() < 0.5: + payment_database.append(order) + print(f"Order processed by Payment Service: {order}") + else: + shipping_database.append(order) + print(f"Order processed by Shipping Service: {order}") + + def retry(self): + print("Retrying queued orders...") + for order in self.queue: + self.execute(order) + self.queue = [] + +# Simulate placing orders +def place_orders(commander, num_orders): + for i in range(num_orders): + order = f"Order {i+1}" + commander.execute(order) + +# Create a Commander instance +commander = Commander() + +# Simulate placing orders +place_orders(commander, 10) + +# Simulate a service recovery scenario +time.sleep(5) +print("Service is back online.") + +# Retry queued orders +commander.retry() +In this example: + +We have two mocked databases (payment_database and shipping_database) to simulate the storage of orders in the payment and shipping services. + +The Commander class handles the distributed transactions. It directs orders to either the payment or shipping service, simulating the unavailability of services in some cases. If a service is unavailable, orders are queued for retry. + +The place_orders function simulates placing orders, and the program attempts to process these orders through the Commander. + +After a simulated service recovery, the retry method is called to process any queued orders. + ## Credits * [Distributed Transactions: The Icebergs of Microservices](https://www.grahamlea.com/2016/08/distributed-transactions-microservices-icebergs/) diff --git a/cqrs/README.md b/cqrs/README.md index 6ac6061ef70e..86bc72aec8e4 100644 --- a/cqrs/README.md +++ b/cqrs/README.md @@ -10,6 +10,62 @@ tag: ## Intent CQRS Command Query Responsibility Segregation - Separate the query side from the command side. +## Explanation +### Real-world example: + +Imagine you are building a complex e-commerce platform. You have a system where customers can place orders, change their account information, and also search for products. In traditional architectures, these operations might all share the same data models and databases. However, as your platform grows, handling these various tasks efficiently becomes a challenge. This is where CQRS comes into play. It allows you to segregate the responsibility for handling commands (e.g., placing orders, updating customer data) from queries (e.g., searching for products). By doing so, you can optimize and scale each side independently to improve overall system performance and manageability. + +### In plain words: + +The CQRS (Command Query Responsibility Segregation) pattern is like having two specialized teams for different tasks. One team, the "command" side, focuses on making changes to the system (e.g., handling orders and updates), while the other team, the "query" side, specializes in retrieving information (e.g., searching and reading data). This separation allows you to tailor each side for its specific role, which can lead to better performance, scalability, and maintainability of your software. + +### Wikipedia says: + +The [CQRS pattern](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation) (Command Query Responsibility Segregation) is an architectural pattern used in software design. It emphasizes separating the responsibility for handling commands (which change the system's state) from queries (which retrieve information without changing state). By doing so, CQRS promotes a more focused and optimized approach to handling different aspects of an application. + +### Programmatic example: +Here's a simplified programmatic example in Python to illustrate the CQRS (Command Query Responsibility Segregation) pattern. In this example, we'll create a basic in-memory "store" where we can issue commands to update data and queries to retrieve data separately: +# Separate data stores for commands and queries +command_store = {} +query_store = {} + +# Command handlers +def create_user(user_id, name): + command_store[user_id] = {"name": name} + +def update_user_name(user_id, new_name): + if user_id in command_store: + command_store[user_id]["name"] = new_name + +# Query handlers +def get_user_name(user_id): + if user_id in query_store: + return query_store[user_id]["name"] + return None + +# Example commands +create_user(1, "Alice") +create_user(2, "Bob") +update_user_name(1, "Alicia") + +# Example queries +query_store = command_store.copy() # Snapshot the command store for querying + +# Retrieve user names +print(get_user_name(1)) # Output: Alicia +print(get_user_name(2)) # Output: Bob +print(get_user_name(3)) # Output: None (user does not exist) + +In this example: + +We have two separate data stores: command_store for handling commands and query_store for handling queries. + +The create_user and update_user_name functions are command handlers responsible for modifying data in the command_store. + +The get_user_name function is a query handler responsible for retrieving data from the query_store. + +We issue commands to create and update users, and then we take a snapshot of the command_store to populate the query_store. This separation of stores demonstrates the CQRS pattern's principle of segregating the responsibility for commands and queries. + ## Class diagram ![alt text](./etc/cqrs.png "CQRS")