Skip to content

Commit

Permalink
"Welcome Guide" which was for armbian-next (#352)
Browse files Browse the repository at this point in the history
* remove all references to Vagrant, as it is no longer supported

* FEL boot is no more

* format mkdocs.yml - no real changes

* The Welcome Guide, or: blergh "What's New" back from `armbian-next`
  • Loading branch information
rpardini authored Mar 8, 2024
1 parent 12d8397 commit b3d71d3
Show file tree
Hide file tree
Showing 7 changed files with 204 additions and 240 deletions.
2 changes: 1 addition & 1 deletion docs/Developer-Guide_Build-Options.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ These parameters are meant to be applied to the `./compile.sh` command. They are
- **SEVENZIP** ( yes | **no** ): create .7z archive with extreme compression ratio instead of .zip
- **BUILD_KSRC** ( **yes** | no ): create kernel source packages while building...
- **INSTALL_KSRC** ( yes | **no** ): ... and pre-install these kernel sources on the image
- **ROOTFS_TYPE** ( **ext4** | f2fs | btrfs | nilfs2 | xfs | nfs | fel ): create image with different root filesystems instead of default `ext4`. Requires setting `FIXED_IMAGE_SIZE` to something smaller than the size of your SD card for `F2FS`
- **ROOTFS_TYPE** ( **ext4** | f2fs | btrfs | nilfs2 | xfs | nfs ): create image with different root filesystems instead of default `ext4`. Requires setting `FIXED_IMAGE_SIZE` to something smaller than the size of your SD card for `F2FS`
- **BTRFS_COMPRESSION** ( lzo | none | **zlib** | zstd ): when choosing `ROOTFS_TYPE=btrfs` select `btrfs` filesystem compression method and compression level. By default the compression is `zlib`.
When selecting `zstd` or setting zlib compression level(`zlib:[1-9]`) user must ensure kernel version is **>=4.14.x**.
When selecting zstd compression level (`zstd:[1-15]`) both the host and the target kernel version must be **>=5.1.x** since kernel started supporting zstd compression ratio only from 5.1 on.
Expand Down
3 changes: 1 addition & 2 deletions docs/Developer-Guide_Build-Preparation.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,10 @@

Not officially supported build environments from community contributions:

- Setting up VirtualBox and compile environment is easy following our [Vagrant tutorial](https://docs.armbian.com/Developer-Guide_Using-Vagrant/),
- [Docker](Developer-Guide_Building-with-Docker.md) environment is also supported for building kernels and full OS images,
- [Multipass](https://gist.github.com/atomic77/7633fcdbf99dca80f31fd6d64bfd0565)

Please note that system requirements (both hardware and OS/software) may differ depending on the build environment (Vagrant, Docker, Virtualbox, native).
Please note that system requirements (both hardware and OS/software) may differ depending on the build environment (Docker, Virtualbox, native).

## How to start?

Expand Down
74 changes: 0 additions & 74 deletions docs/Developer-Guide_FEL-boot.md

This file was deleted.

85 changes: 0 additions & 85 deletions docs/Developer-Guide_Using-Vagrant.md

This file was deleted.

127 changes: 127 additions & 0 deletions docs/Developer-Guide_Welcome.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Welcome to `armbian/build`, the Armbian build system

Welcome to the Armbian build system (`armbian/build`).
The build system has significantly changed since release 23.02, when the `armbian-next` effort was merged into
our `main` branch.
Please expect some rough edges and report them to us.

## Errors, errors everywhere

Previously, when faced with an error, the build system would simply ignore them, hardly log them anywhere, and continue.
This led to builds completing successfully, but with (sometimes very sneakily) broken packages and images.
Get used to seeing errors stopping builds now. Inspect the whole log, specially the errors, the last few lines, and the
stack trace.
We try, as much as possible given Bash limitations, to show the source of the error, with a "stack trace" of sorts.
The topmost elements of the stack trace are usually the most relevant. Each line is prefixed with the file and line
number where the error occurred.
Open the file and go to the line number to see the source of the error. Most times, this simple inspection is enough to
understand the error.

### (ANSI) Logging

Logging is a bit more structured now. Logs are output to `output/logs`, in a few different formats.
We output ANSI color both to the screen and to the logs.
_Please_ add `SHARE_LOG=yes` to share your logs with us when reporting issues, that allows us to check the logs on a web
browser and keep to correct formatting.

## Command line syntax has changed

General CLI syntax: `./compile.sh PARAM=value OTHER_PARAM=other_value [<configfile> <configfile> ...] [<command>]`

- where `command` defaults to `build` if you don't specify it; could also be `kernel-config` or `u-boot` etc
- you _can't_ have a config file with the same name as a possible `<command>` (system will check & bomb if so)
- also: there's no more `default` config -- you have to be explicit
- also: there's no more `docker` config -- Docker is fully auto-managed now. The system will complain if you have one.
- you can specify `PARAM=value`, `<configfile>` or `<command>` in _any order_

## No more `config-default.conf`, you need to specify the config file in the command line

- No "default" config is auto-loaded anymore. Default config lead to unreproducible failing builds and was a source of
confusion.
- The configs still go to the same place, `userpatches/config-xyz.conf` -- but now you gotta tell the build system to
use that config, like `./compile.sh BOARD=xxx xyz`; otherwise works the same.

## Artifacts, cache, what the ...?

The `armbian/build` system is currently undergoing refactoring to improve its structure. Previously, the build system
was a single, complex Bash script that mixed the building of `.deb` packages with the creation of images.

This was reworked into a `1-to-N` image-to-artifact dependency tree; a certain image build will depend on N possible
"artifacts". Artifacts are either `.deb` packages, a `.tar` of multiple `.deb` packages, or a `rootfs.tar.zstd`. Each
artifact can be individually built, and has a specific name and a _version_.

Each artifact is also now **cached by default** using OCI storage at ghcr.io (GitHub Container Registry). To achieve
_consistent caching_, each artifact produces a version that includes _hashes_ of its composing files, variables,
patches,
hooks, external git SHA1 references, etc. That way we can consistently check the remote OCI cache for previously-built
artifacts, and possibly save image builders from having to build heavy packages just to produce an image.

### TL-DR about artifacts and caching:

- `KERNEL_ONLY=yes` and `KERNEL_ONLY=no` are no more. Use the `kernel` CLI command instead.
- `ARTIFACT_IGNORE_CACHE=yes` can help if you're getting false positives. Please also report the problem, with a
complete logfile.

## Automatic Docker/sudo launcher

- `compile.sh` will prefer to use Docker if it detects Docker is installed and working.
- This handles Docker Desktop and Rancher Desktop (in Docker emulation mode) under macOS/Darwin, including Apple
M1/M2.
- You **don't need and actually can't have the old docker config file**.
- If Docker is not installed, it will try to use `sudo` to run the build as root.
- If you run directly as root, it will warn you and ask you to run it without sudo.

## Kernel Git Trees: shallow vs full

During the build, depending on which local or remote caches are hit, it might be needed to build the Linux Kernel.

The kernel's git repo is huge. Most build systems resort to fetching "shallow" trees directly from upstream git servers,
to save bandwidth. Unfortunately that creates immense extra CPU load on the git servers. To avoid this problem,
Armbian produces daily automated git tree exports cached in ghcr.io OCI repositories, and only uses `git fetch` to
update the relatively small new changes from the upstream git server.

There are two types of cached Kernel git trees:

- `full` is a complete git tree, including all of Torvald's `master` and all of the currently-supported `stable`
branches.
- `full` is very large download and requires a lot of disk space.
- `full` is more useful over time and when building multiple different kernels on the same machine, like for CI
servers or developer workstations.
- `shallow` is a shallow tree for a specific `stable` branch
- `shallow` is a much smaller download and requires less disk space
- `shallow` is appropriate for restricted devices like SBCs which will build a single kernel

**TL,DR: ** `KERNEL_GIT=full` or `KERNEL_GIT=shallow` or let the system decide for you.

## Consider Forking before cloning the repo

Before cloning the repo, consider forking it first. This will allow you to make changes and submit pull requests.
You will need a GitHub account to do this;
see [GitHub's documentation](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) for more
information.
If you fork, make sure to keep your fork up-to-date with the main repo, by rebasing your fork.

## Some really confusing stuff still remains

This is (by far) not a complete list:

- wifi/other kernel drivers are still using pre-armbian-next code, and are very hard to work with. it's not only the
contents are a mess, the way the whole thing works leads to more and more compounding work. To make it worse, family
patches sometimes need to patch driver code, leading to a cycle of sadness for developers. We're still coming up with
a plan to completely replace this lest most of us go insane.
- although "aggregation" has been rewritten in Python, it still mostly works using the legacy principle, by scanning
directories and files in a very complex and error-prone way. This is a source of many bugs and confusion. We plan to
replace this with pure extensions eventually.
- "minimal" images are far from really minimal. It is common to see "extensions" that actually remove stuff put there
by the core. Those should be refactored.
- board-side scripts (armbian-config, firstrun, hwoptimization, etc) haven't really changed with armbian-next, and are
in dire need of a rewrite. They're also a source of many bugs and confusion. They also need to be extensible.
- we've mostly working Kernel headers (linux-headers pkg) for 5.10+ including some vendor kernels; no kernel-headers for
4.x kernels. sorry. We've no plans to support this.

## Multiple u-boot's for same board

Hmm. This is an interesting problem. We can build u-boot twice, using `UBOOT_TARGET_MAP`.
Some example I did in https://github.com/armbian/build/blob/main/config/boards/odroidhc4.conf#L15-L20 may help.


2 changes: 0 additions & 2 deletions docs/Release_Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -1766,7 +1766,6 @@ Build script:
* added option to create minimal images with around 500Mb in size BUILD_MINIMAL="yes"
* added initial support of MCIMX7SABRE board (CSC)
* updates for xt-q8l-v10 (CSC)
* vagrant-disksize is beeing determined automatically
* Docker is installed automatically if one want use it (Debian based build host only)
* refactor build all images scripting that images can be build in full parallel mode
* added one file for storing which combinations shell be made for each board
Expand Down Expand Up @@ -2523,7 +2522,6 @@ Build script:
* added docker as host
* Added Belink X2 (H3 based media player), and Roseapple (S500) as WIP target
* introducted CLI_TARGET per board
* prepared FEL boot
* prepared Xenial target
* fixed USB redirector building on all kernels
* support for Xenial as a build host is 95% ready.
Expand Down
Loading

0 comments on commit b3d71d3

Please sign in to comment.