Use modules-file instead of generating it from config.mk and feeds.conf #781

SvenRoederer · 2020-04-10T07:09:41Z

The modules-file controls which code of the upstream-repos is used. With the implementation of PR #714 we added some "glue-code" to create the modules file from the currently used files (config.mk, feeds.conf).
This "glue-code" works well with the official Upstream-repos but seems to have some problems with alternative repos as shown by issue #780. This PR drops the "very brittle" code and moves the repo-definitions to modules directly.

Drops the additional converting code
speeds up build, by not copying each repo an additional time to find the branch of the referencing commit
moves the definition of any upstream-repo into a single file (not spread around two files anymore)
an instant lookup of the branch, the code firmware-build is based on, is possible

pmelange · 2020-04-12T16:57:11Z

I am currently building with these changes cherry-picked into a private branch of the "next" branch. The goal is to prototype kernel 5.4 on the erx-sfp.

One thing that I noticed, is that I have to put a commit ID into the modules file. If not, I get the following error:

Initialized empty Git repository in /home/pmelange/src/mt7621/openwrt/.git/     
remote: Enumerating objects: 2, done.                                           
remote: Counting objects: 100% (2/2), done.                                     
remote: Compressing objects: 100% (2/2), done.                                  
remote: Total 488150 (delta 0), reused 0 (delta 0), pack-reused 488148          
Receiving objects: 100% (488150/488150), 171.31 MiB | 5.21 MiB/s, done.         
Resolving deltas: 100% (332298/332298), done.                                   
From https://github.com/vDorst/openwrt                                          
 * branch            ramips-5.4 -> FETCH_HEAD                                   
fatal: Not a valid object name: ''.                                             
unable to find commit "" on branch "ramips-5.4" in repo "https://github.com/vDor
st/openwrt.git".                                                                
make: *** [Makefile:90: .stamp-pre-patch] Error 1

It would be really cool if this code would use the HEAD of the branch if no commit-id is given.

So, I would say that this fixes issue #780, but not #477

pmelange · 2020-04-13T09:16:54Z

Running "make clean" deletes the modules file. Afterwards running make again results in

rm -f /home/pmelange/src/mt7621/openwrt/.config
rm -f .stamp-
make: *** No rule to make target '/home/pmelange/src/mt7621/modules', needed by '.stamp-pre-patch'.  Stop.

Also, it seems like that if I change the commit-id (for example in luci) in the modules file, and run make again, then the new commits are not integrated and built.

SvenRoederer · 2020-04-13T12:02:21Z

fixed in 3443833

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84.

SvenRoederer · 2020-04-25T10:57:42Z

Any additional feedback?
What about merging this?

pmelange · 2020-04-25T13:22:17Z

I still haven't gotten around to testing alternative git repos. Specifically I would like to try the equivalent of src-lnk, which I have found to be very useful for prototyping

Akira25 · 2020-04-25T13:26:00Z

I havn't test this, but I like the goals to be achieved by this PR. Sounds like a good Idea.

pmelange · 2020-04-25T16:00:13Z

I'm having trouble being able to use alternative git feed types. How are we supposed to be able to use the different types of feeds support by openwrt and shown here: https://openwrt.org/docs/guide-developer/feeds

SvenRoederer · 2020-04-25T21:09:50Z

this PR is only about solving the problem described in #780 (interpreting external GIT repos from config.mk and feeds.conf to build the modules-file). In addition this PR should also fix #772.
We should avios to use a solution for an issue (here #780) to solve another different issue (#777 ) too. Even both issues somehow relate to each other.

SvenRoederer · 2020-04-25T21:14:19Z

I'm having trouble being able to use alternative git feed types. How are we supposed to be able to use the different types of feeds support by openwrt and shown here: https://openwrt.org/docs/guide-developer/feeds

@pmelange can you put this into a separate issue?

pmelange · 2020-04-26T08:30:43Z

I'm having trouble being able to use alternative git feed types. How are we supposed to be able to use the different types of feeds support by openwrt and shown here: https://openwrt.org/docs/guide-developer/feeds

@pmelange can you put this into a separate issue?

See issue #789

SvenRoederer · 2020-04-26T16:37:47Z

Independent of fix for issue #789 this PR will solve the #780 & #772 already.

SvenRoederer · 2020-05-06T22:04:23Z

I'm going to merge this by this week, as it solves to issues and doesn't break anything. I tested it for some time in the "SAm0815-experimental" branches.

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR #781 * Issue #780 * Issue #772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR #781 * Issue #780 * Issue #772

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR #781 * Issue #780 * Issue #772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR #781 * Issue #780 * Issue #772

SvenRoederer · 2020-05-08T09:03:05Z

merged into master with 487cb33 and c6aa875.

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR #781 * Issue #780 * Issue #772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR #781 * Issue #780 * Issue #772

Switch from defining the upstream-repos and relating commit in config.mk and feeds.conf to define all in the modules-file. The modules-files is the real file used by the build-system to setup the working checkout. It originates from the Gluon-buildsystemand some documentation is avail at https://gluon.readthedocs.io/en/latest/dev/build.html This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84. This also relates to * PR freifunk-berlin#781 * Issue freifunk-berlin#780 * Issue freifunk-berlin#772

55a8827 batman-adv: update to version 2022.0 911a4ba batctl: update to version 2022.0 3aa4cfb alfred: update to version 2022.0 d7adf18 Merge pull request freifunk-berlin#781 from ecsv/batadv-2022.0

SvenRoederer added the buildsystem label Apr 10, 2020

This was referenced Apr 10, 2020

buildenv: using an forked openwrt repo and branch from github fails #780

Closed

buildenv: using HEAD of a feed will not work #777

Open

Build process fails: nchname.txt.txt: No such file or directory #772

Closed

SvenRoederer force-pushed the modules-file branch from 789c95f to 3443833 Compare April 13, 2020 12:01

SvenRoederer added 2 commits April 13, 2020 14:08

Makefile: drop the "glue-code" for creating the modules-file

aed5088

When using the modules-file in place of config.mk and feeds.conf to define the upstream-repos, there is no need to keep the code which will synthesize it. This primarily reverts 7a72a84.

SvenRoederer force-pushed the modules-file branch from 3443833 to aed5088 Compare April 13, 2020 12:10

SvenRoederer changed the title ~~Use modules-file~~ Use modules-file instead of generating it from config.mk and feeds.conf Apr 25, 2020

pmelange mentioned this pull request Apr 26, 2020

buildenv: new "modules" file based feed syntax does not allow for alternative feed types #789

Open

SvenRoederer closed this May 8, 2020

SvenRoederer deleted the modules-file branch May 8, 2020 09:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use modules-file instead of generating it from config.mk and feeds.conf #781

Use modules-file instead of generating it from config.mk and feeds.conf #781

SvenRoederer commented Apr 10, 2020

pmelange commented Apr 12, 2020

pmelange commented Apr 13, 2020

SvenRoederer commented Apr 13, 2020

SvenRoederer commented Apr 25, 2020

pmelange commented Apr 25, 2020

Akira25 commented Apr 25, 2020

pmelange commented Apr 25, 2020

SvenRoederer commented Apr 25, 2020

SvenRoederer commented Apr 25, 2020

pmelange commented Apr 26, 2020

SvenRoederer commented Apr 26, 2020

SvenRoederer commented May 6, 2020

SvenRoederer commented May 8, 2020

Use modules-file instead of generating it from config.mk and feeds.conf #781

Use modules-file instead of generating it from config.mk and feeds.conf #781

Conversation

SvenRoederer commented Apr 10, 2020

pmelange commented Apr 12, 2020

pmelange commented Apr 13, 2020

SvenRoederer commented Apr 13, 2020

SvenRoederer commented Apr 25, 2020

pmelange commented Apr 25, 2020

Akira25 commented Apr 25, 2020

pmelange commented Apr 25, 2020

SvenRoederer commented Apr 25, 2020

SvenRoederer commented Apr 25, 2020

pmelange commented Apr 26, 2020

SvenRoederer commented Apr 26, 2020

SvenRoederer commented May 6, 2020

SvenRoederer commented May 8, 2020