This is the code repository for In-Memory Analytics with Apache Arrow, published by Packt.
Perform fast and efficient data analytics on both flat and hierarchical structured data
Apache Arrow is designed to accelerate analytics and allow the exchange of data across big data systems easily. In-Memory Analytics with Apache Arrow begins with a quick overview of the Apache Arrow format, before moving on to helping you to understand Arrow’s versatility and benefits as you walk through a variety of real-world use cases.
This book covers the following exciting features:
- Use Apache Arrow libraries to access data files both locally and in the cloud
- Understand the zero-copy elements of the Apache Arrow format
- Improve read performance by memory-mapping files with Apache Arrow
- Produce or consume Apache Arrow data efficiently using a C API
- Use the Apache Arrow Compute APIs to perform complex operations
- Create Arrow Flight servers and clients for transferring data quickly
- Build the Arrow libraries locally and contribute back to the community
If you feel this book is for you, get your copy today!
All of the code is organized into folders.
The code will look like the following:
...
// add these imports
"fmt"
"github.com/apache/arrow/go/v8/arrow/arrio"
...
Following is what you need for this book: This book is for developers, data analysts, and data scientists looking to explore the capabilities of Apache Arrow from the ground up. This book will also be useful for any engineers who are working on building utilities for data analytics and query engines, or otherwise working with tabular data, regardless of the programming language. Some familiarity with basic concepts of data analysis will help you to get the most out of this book but isn't required. Code examples are provided in the C++, Go, and Python programming languages.
With the following software and hardware list you can run all code files present in the book (Chapter 1-11).
| Software required | OS required |
|---|---|
| An internet connected computer | |
| Git | Windows, Mac OS X, and Linux (Any) |
| C++ compiler capable of C++11 or higher | Windows, Mac OS X, and Linux (Any) |
| Python 3.7 and higher | Windows, Mac OS X, and Linux (Any) |
| Conda (optional) | Windows, Mac OS X, and Linux (Any) |
| vcpkg (optional) | Windows |
| MSYS2 (optional) | Windows |
| CMAKE 3.5 or higher | Windows, Mac OS X, and Linux (Any) |
| make or ninja | Mac OS X, and Linux (Any) |
| Docker | Windows, Mac OS X, and Linux (Any) |
| Go 1.16 or higher | Windows, Mac OS X, and Linux (Any) |
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. Click here to download it.
Matthew Topol is an Apache Arrow contributor and a Staff Software Engineer at Voltron Data. He joined FactSet Research Systems, Inc. in 2009, working in both infrastructure and application development, led development teams, and architected large-scale distributed systems for processing analytics on financial data. He recently joined Voltron Data to work full time on the Apache Arrow libraries directly. In his spare time, Matt likes to bash his head against a keyboard, develop and run delightfully demented games of fantasy for his victims—er—friends, and share his knowledge with anyone interested enough to listen.
If you have already purchased a print or Kindle version of this book, you can get a DRM-free PDF version at no cost.
Simply click on the link to claim your free PDF.