Commit Graph

80131 Commits

Author SHA1 Message Date
Deepesh Varatharajan
b189b2fb86 rust-target-config: PPC64 targets require explicit ABI selection to avoid build failures with rustc.
Without a specified ABI, rustc panics with the following error:
|   thread 'rustc' panicked at compiler/rustc_codegen_ssa/src/back/metadata.rs:394:21:
|   No ABI specified for this PPC64 ELF target.

This issue was occuring because of the following Rust commit:
9c1180b623

As noted in the upstream changes:
If the flags do not correctly indicate the ABI,
linkers such as ld.lld assume that the ppc64 object files are always ELFv2,
which leads to broken binaries if ELFv1 is used for the object files.

Because of this, it is now required to explicitly specify the ABI for PPC64 targets
using one of the following:
"elfv1" => EF_PPC64_ABI_ELF_V1,
"elfv2" => EF_PPC64_ABI_ELF_V2,

If no ABI is specified, the Rust compiler will panic with the error:
No ABI specified for this PPC64 ELF target

To address this:
- Set 'elfv2' for powerpc64le (little-endian), which mandates ELFv2 ABI.
- Set 'elfv1' for powerpc64 (big-endian), which defaults to ELFv1 ABI.

(From OE-Core rev: 6cee30b7941c22eef52011b6bac0d3c0d7944abe)

Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:28:56 +01:00
Khem Raj
4dabfb3f21 libpam: Fix build with LLD linker
LLD is strict about versioned symbols unlike BFD linker, it flags
undefined ones, Allow undefined symbols like BFD linker

Fixes
| riscv64-yoe-linux-ld.lld: error: version script assignment of 'global' to symbol 'pam_sm_acct_mgmt' failed: symbol not defined
| riscv64-yoe-linux-ld.lld: error: version script assignment of 'global' to symbol 'pam_sm_chauthtok' failed: symbol not defined
| riscv64-yoe-linux-ld.lld: error: version script assignment of 'global' to symbol 'pam_sm_close_session' failed: symbol not defined
| riscv64-yoe-linux-ld.lld: error: version script assignment of 'global' to symbol 'pam_sm_open_session' failed: symbol not defined
| riscv64-yoe-linux-clang: error: linker command failed with exit code 1 (use -v to see invocation)

(From OE-Core rev: a53fddbb99dc746439b5b2adfb7f747ba25a856f)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:27:19 +01:00
Johannes Schneider
7320e59aec bitbake: bitbake-setup: commandline: use subsubparser for settings {list,set,unset}
Previously the sub-command 'settings' would take any number of
arguments and then silently do nothing if the number wasn't three.

The help text was also not clear about this, marking the positionals
separately as optional:

usage: bitbake-setup settings [-h] [--global] [--unset UNSET UNSET] [-l] [section] [key] [value]

The '--unset SECTION SETTING' also did not  integrate too well, as it
had its own positional arguments for section+setting.

For a bit more consistency and a explorable help, a sub-subparser is
added, that provides the commands:
  bitbake-setup settings list
  bitbake-setup settings set foo bar baz
  bitbake-setup settings unset foo bar
with a '--global' that is added from a stand-alone parent parser, so
that it shows up in all sub-command help texts.

The new help text now reads:
usage: bitbake-setup settings [-h] [--global] {list,set,unset} ...

and the respective sub commands:
usage: bitbake-setup settings list [-h] [--global]
usage: bitbake-setup settings set [-h] [--global] <section> <setting> <value>
usage: bitbake-setup settings unset [-h] [--global] <section> <setting>

(Bitbake rev: 8b582ef8dd0cef0192d4c0104bcd9b5d642d132c)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
897f3020da bitbake: bitbake-setup: add 'metavar' for self-descriptive parameters
Add a metavar to the argparse options to have a self-descriptive help text.
Otherwise argpase defaults to use the argument name in all-uppercase.

Before:

usage: bitbake-setup [-h] [-d] [-q] [--color COLOR] [--no-network] [--global-settings GLOBAL_SETTINGS] [--setting SETTING SETTING SETTING]
                     {list,init,status,update,install-buildtools,settings} ...

After:

usage: bitbake-setup [-h] [-d] [-q] [--color COLOR] [--no-network] [--global-settings PATH] [--setting SECTION SETTING VALUE]
                     {list,init,status,update,install-buildtools,settings} ...

(Bitbake rev: 83cecc9356a0684f90249d527fe372298ae92719)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
12d456bc9b bitbake: bitbake-setup: use args.cmdline_settings for --settings
To align the storage destination with the internally used variable
name. This makes room for having another option use 'args.setting'

(Bitbake rev: 14d8535309abc78ee30cfdb51bba2e00b474f443)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
0a5c5430c5 bitbake: bitbake-setup: init: suggest removing a partially initialized top-dir
In cases where the first call to 'init' failed or was aborted before
creating the 'build/init-build-env' has been created, a user can get
stuck: a second call to init aborts, suggesting 'status' or 'update'
but these to refuse because the --build-dir is not valid.

Guide the user by adding a suggestion to start over from scratch.

(Bitbake rev: 11b2740c3e19e0c6680229c6bbce3691c73746a8)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
d81884b3a9 bitbake: bitbake-setup: correct 'setting' to 'settings' in a couple of help texts
(Bitbake rev: c5e04ce986b680f39f6c7851e4236c4afa4ac3f4)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
cd4fe0aada bitbake: bitbake-setup: further rework the settings handling
After some further feedback, additional changes are made:

1. 'setting' command is renamed to 'settings' to better reflect
that it is an interface to various ways of managing settings.

2. This command now has a -l/--list option to list all settings
with their values (same as 'git config -l').

3. A new level of settings (built-in defaults) is added,
and used as a last resort after command line options, top dir
settings file and global settings file.

4. This means bitbake-setup does not have to write and use a
global settings file, and it no longer does so when initializing
a build, avoiding default 'pollution' of ~/.config/bitbake-setup/
which can be problematic or unwelcome.

A global settings file is still created if a setting is explicitly
requested to be placed into it.

5. 'install-global-settins' is removed as the use case for it
(tweak default settings before using them to initialize a build)
can be achieved by setting the settings individually.

5. Similarly, a top dir settings file is no longer created by default
and only appears if a setting needs to be written into it.

6. Default dl-dir is again created inside a top directory and not
in ~/.cache/ to make default builds fully contained in the top
directory (which was also asked about).

(Bitbake rev: 664f8ec48d42d2ddc5f234c4f7d590fa597f489a)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
401e7b6a10 bitbake: bitbake-setup: rework the settings handling
This is the outcome of various discussions, suggestions and pull
requests on github.

What has specifically changed?

1. The sources for the settings are no longer separated, but are stacked and given priorities,
from highest to lowest:

    a. '--setting section key value' on the command line

    b. a settings file in the top directory

    c. a global settings file in ~/.config/bitbake-setup/ (or in a file pointed to by --global-settings)

Any setting can be in any of these three locations (other than top dir name and prefix which do not
make sense in the settings file in the top directory).

2. A global settings file must contain all of the needed settings, while a settings file
in the top directory can be empty (and this is how they are written out if they do not exist).

Specifically, both dl-dir and registry settings have been relocated to the global file,
and dl-dir defaults to ~/.cache/bitbake-setup/downloads, rather than somewhere in top dir.

3. The file name for both global and top dir settings is now 'settings.conf'.

4. --top-dir-prefix and --top-dir-name options have been removed and superseded by
a generic, universal --setting option.

5. 'install-settings' command has been removed, as it is no longer does anything useful,
and is superseded by the 'setting' command (see below).

'install-global-settings' has been retained, to be able to have a set of global defaults
that can be changed without initializing a build.

6. 'change-setting', 'change-global-setting' and 'install-settings' have all been replaced
by a single 'setting' command that mimics 'git config' in its parameters:

    a. Changing a setting: bitbake-setup setting [--global] default dl-dir /path/to/downloads

    b. Removing a setting: bitbake-setup setting [--global] --unset default dl-dir

(Bitbake rev: 713e7f213c6d4a620be9ce34d5f4396af48e1d69)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
e9345af4f8 bitbake: bitbake-setup: tests: add environment-passthrough
Add a test configuration to cover the 'bb-env-passthrough-additions'
conf.json key, and add it to the test routine.

(Bitbake rev: 24f12b68692f9ebb5d3813bc3b1771e43298f640)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
151a0f6361 bitbake: bitbake-setup: tests: add a template-only test configuration
bitbake-setup conf.json can be purely template driven, to setup older
yocto LTS (e.g. scarthgap) based layer-collections - where fragment
support was not yet present in oe-core+bitbake.

Add a configuration for that scenario and add it to the test routine.

(Bitbake rev: 23e121befa0779fbb1f342984b583c04ccc637a0)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Johannes Schneider
60d995ccc4 bitbake: bitbake-setup: support adding environment-passthroughs to the init-build-env
This patch adds support for extending the BB_ENV_PASSTHROUGH_ADDITIONS
environment variable from within the `init-build-env` wrapper script -
generated by either oe-core's oe-setup-build, or by `bitbake-setup` -
based on per-configuration JSON settings.

This enables CI workflows to inject environment-specific data - such
as build number, host, build type, or credentials required to fetch
from certain SRC_URIs - which cannot be captured via configuration
fragments alone. These variables are now handled early in the setup
process and exported directly into the build environment.

Example:

  "bb-env-passthrough-additions": [
      "ACME_DIR",
      "ARTIFACTORY_TOKEN",
      "ARTIFACTORY_USERNAME",
      "GITHUB_TOKEN",
      "GITHUB_PROTOCOL",
      "KEY"
  ]
  <snip>

the resulting 'init-build-env' would then be:
  # environment passthrough added by bitbake-setup
  export BB_ENV_PASSTHROUGH_ADDITIONS=" \
  $BB_ENV_PASSTHROUGH_ADDITIONS \
  ACME_DIR \
  ARTIFACTORY_TOKEN \
  ARTIFACTORY_USERNAME \
  GITHUB_TOKEN \
  GITHUB_PROTOCOL \
  KEY"
  # init-build-env wrapper created by bitbake-setup
  . /tmp/acme_master-acme-distro_acme-machine_bang/layers/openembedded-core/oe-init-build-env /tmp/bitbake-setup/gs/acme_master-acme-distro_acme-machine_bang/build

(Bitbake rev: 782ab99e7a04fba43bdcf5763a6280785944ae3f)

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
39af683c76 bitbake: bitbake-setup: add support for skipping a fragment selection
In autobuilder testing a use case arised where
- the available choices in configuration file for choosing a machine are incomplete
- putting every possible machine choice into that configuration is undesirable/not possible
- autobuilder code can write a machine selection into the bitbake config
after the fact.

So this --skip-selection option is intended for advanced users that know what they're doing
and is generally not recommended as it requires manually tweaking the bitbake config to
make it usable.

(Bitbake rev: 8cb2372bdad381179969d2ecbba7decaf03a7c5f)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
28055e0c0d bitbake: bitbake: registry: make a separate configuration for poky with sstate mirror
Using sstate mirror places much higher requirements on both network robustness
and network bandwidth than just fetching source code. When the network doesn't meet
those, the user experience can be very frustrating as errors can be cryptic and
intermittent, or bitbake would just seemingly do nothing for a long time.

Let's make sstate an opt-in, and provide a caution for using it. When it does
work, it works wonderfully and does accelerate builds significantly.

(Bitbake rev: fe88a8e9cd5ccbc26508cf524a2f71a06d3df03a)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Paul Gortmaker
3fe3e0ba1f bitbake: bitbake-setup: clarify that default answer to prompts is no
It is common practice to put the default choice in upper case for
yes/no interactive prompts, so that when people just hit enter, they
know what they are getting.  An example that linux users are probably
familiar with is "sensors-detect" from the "lm-sensors" package.

Unify all the prompts to be the same and indicate that the default
answer from hitting enter is a no with an upper case N.  No functional
changes.

(Bitbake rev: 7d6225722e21b116ae164fbaae2a918534a5107b)

Signed-off-by: Paul Gortmaker <paulg@kernel.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Yoann Congal
20f58af1b3 bitbake: bitbake-setup: allow using {THISDIR}/my-layer
This implement the ability to use "{THISDIR}/my-layer" in the
"bb-layers" list. "{THISDIR}" is remplaced by the directory containing
the configuration file.

In small projects, we try to keep the setup a simple as possible: a
single git repo containing both the build confguration (e.g.
a bitbake-setup configuration file) and the meta layer with project
recipes/machine/distro.

This change allows this kind of setup:
├── meta-my-project/ # the project layer
└── my-project.conf.json   # the bb-setup configuration file

by writing, in my-project.conf.json:
  "bitbake-setup": {
    "configurations": [{
      "bb-layers": [
        "{THISDIR}/meta-my-project"

Note: in this case meta-my-project is not present as a "source", so, not
handled by bb-setup update/status. It is expected of the user to handle
this on their own (is our case, a simple git workflow).

(Bitbake rev: b3153be29de8b8570b0c184369bd41f4c646cf92)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
9f820ce0f4 bitbake: lib/bb/tests/setup.py: unset BBPATH to ensure isolation from the existing bitbake environment
bitbake-setup deduces top directory from BBPATH, which, if set, interferes with
the tests' own setup.

(Bitbake rev: e974d42eb5229f755e14b46a00ad06b23b53e143)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
bebbf40426 bitbake: lib/bb/tests/setup.py: define test parameters in a single dictionary
This makes maintaining and extending them easier.

(Bitbake rev: 16dc8e3dad7dde7e7651cce13549e61574cafba1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Yoann Congal
0c37775349 bitbake: bitbake-setup: dash support for init-build-env script
Being minimalist, dash does not support the (non-POSIX) feature of
passing an argument while sourcing a script. Like in
  . <some path>/oe-init-build-env <build dir>

With dash, one must use:
  set <build dir>       # puts <build dir> in $1
  cd <some path>
  . ./oe-init-build-env # can only be called from its directory in dash

To do this:
* Instead of a symlink to oe-init-build-env, keep a symlink to the
  directory containing it (called "oe-init-build-env-dir")
* Generate a init-build-env script that dash can source using the above
  snippet.

(Bitbake rev: 442b41c7949e1522212b66b16811f6b64b089b23)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Yoann Congal
47f6bd30b4 bitbake: bitbake-setup: suggest "." instead of "source"
"." is in POSIX standard[0], whereas "source" is only supported in more
feature-full shells (bash, zsh, ...)

[0]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#dot

(Bitbake rev: 22c5fe7b2de74841e86d28a81143bd1a717518d9)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
32a3828d59 bitbake: bitbake-setup: improve robustness of loading/writing settings
Particularly:

- ensure global settings command line argument is always expanded to
full path
- ensure any errors that happen when loading settings are reported
at that point, otherwise we get an empty dictionary and cryptic
key errors later

(Bitbake rev: 578afa2f05dfa6727952365918df703875070f64)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
1a55c45617 bitbake: bitbake-setup: add support for specifying branches in repo checkouts
Previously bitbake-setup was checking out 'detached commits' using
fetcher's nobranch feature, as that is the only option when only a revision is in the config.

Branches are optional, but beneficial, as

- checkout directory will be on a branch, making it easier for users
to understand where they are if they need to make changes (also
bitbake will print branch information instead of saying 'HEAD:sha').

- supply chain security! Enforcing a branch means any specified revision
has to be on it, and no one can sneak in (accidentally or deliberately!)
some dangling commit, or something from their private branch in the same repo.

(Bitbake rev: 45ed9b9faebdaa8cb7cc8dd2a6d51ec8eea06e73)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Richard Purdie
dd358f75f4 bitbake: bitbake-setup: Improve the already initialized test
If the directory already exists but hasn't been setup, the current test
can fail so improve it.

(Bitbake rev: dac27bd5acbde1807b9637f809fd0ee5bf424286)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Richard Purdie
3d48b2a7b1 bitbake: bitbake-setup: Switch to internal default registry files
Switch the url to be the default internal registry rather than a private
repo which was intended for testing.

(Bitbake rev: e031b75b5b92552d812d2305a35ce90eb6c68b78)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Richard Purdie
f2b06607d3 bitbake: Add default registry config for OpenEmbedded's nodistro and Yocto Project's poky
Rather than forcing every user to have to access a separate repository
for data for some key defaults, add those for nodistro and poky. This
gives us some commonly used setups and also something to test against too.

We will need to come up with a criteria for adding these, most likely based
on community usage/demand with the OE TSC governing that policy.

(Bitbake rev: 562de93d5b0e7cf0a9a43714d0563e5bcf6e9931)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Richard Purdie
9c80f22d37 bitbake: bitbake-setup: Allow local registry paths
It is useful for bitbake-setup to support local paths without access through
the fetcher so that internal data to the bitbake repository can be used as
a default.

(Bitbake rev: ec82a6d402a0bec5704310c66c6f4206c75e4fc9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
4e94ef6f0b bitbake: cooker: adjust the error message about missing websockets module to mention bitbake-setup
(Bitbake rev: 2d15e6fc93f6a73c9c84e399613f931f1c783922)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
266379f0be bitbake: bitbake-setup: add 'install-buildtools' command
This basically calls install-buildtools from oe-core/poky, but
it ensures via command line parameters that the installation
location is stable and the downloads are preserved for reproducibility:

$ bin/bitbake-setup install-buildtools
Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf

======
Buildtools archive is downloaded into /home/alex/bitbake-builds/yocto-master-testing/buildtools-downloads/20250319141333 and its content installed into /home/alex/bitbake-builds/yocto-master-testing/buildtools

... (output from install-buildtools script)
======

It also detects when buildtools are already installed, and will direct
users what to do:
======
alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup install-buildtools
Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf

Buildtools are already installed in /home/alex/bitbake-builds/yocto-master-testing/buildtools.
If you wish to use them, you need to source the the environment setup script e.g.
$ . /home/alex/bitbake-builds/yocto-master-testing/buildtools/environment-setup-x86_64-pokysdk-linux
You can also re-run bitbake-setup install-buildtools with --force option to force a reinstallation
======

This commits includes fixes by Gyorgy Sarvari <skandigraun@gmail.com>
https://github.com/kanavin/bitbake/pull/2

(Bitbake rev: 3fe3096847046110c72b23fce37fb4a459b1d748)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
a66c929a6a bitbake: bitbake-setup: add tests to bitbake-selftest
Run like this:

alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-selftest -v bb.tests.setup
test_setup (bb.tests.setup.BitbakeSetupTest.test_setup) ... ok

----------------------------------------------------------------------
Ran 1 test in 9.223s

OK

The test does a basic run-through of init, then status/update
on an unchanged configuration, then status/update on a
configuration changed via new commits to the test layer,
then status/update on configuration changed via the top
level json config file.

Note that nothing whatsoever is fetched from the network;
the test relies entirely on synthetic data contained inside
itself, including minimal stubs for oe-setup-build and
bitbake-config-build. This data is used to create temporary
git repositories then clone them via local filesystem URIs.

Later on this can be supplemented by an oe-selftest that
tests bitbake-setup against real config files in the
official configuration repository and real layers,
templates and fragments.

(Bitbake rev: e3aa3eb46bd3196fa5415fa36e3737636fd6a1c0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Alexander Kanavin
b05d3c8a31 bitbake: bitbake-setup: add the initial implementation
Preamble
========

The latest iteration of this patchset is available at
https://github.com/kanavin/bitbake
I recommend taking the patches from there to ensure that
you are not trying out outdated code.

For the rationale and design guidelines please see this message:
https://lists.openembedded.org/g/openembedded-architecture/message/1913

Left out for now but will be done later:

- official configuration repository

- documentation

Amble *scratch* HOWTO
=====================

1. If you don't know where to start, run 'bitbake-setup init'.

Bitbake-setup will ask a few questions about available configuration choices and set up a build.

Note: 'init' sub-command can also take a path or a URL with a configuration file directly.
You can see how those files look like here:
https://github.com/kanavin/bitbake-setup-configurations

2. You can then source the bitbake environment and run bitbake to perform builds as usual:

$ . /home/alex/bitbake-builds/yocto-master-options-poky-distro_poky-machine_qemux86-64/build/init-build-env

Also, subsequent status/update commands will not require a separate --build-dir argument telling
bitbake-setup where the build is.

3. To check if the build configuration needs to be updated, run:

===
$ bin/bitbake-setup status
...
Configuration in /home/alex/bitbake-builds/poky-alex/ has not changed.
===

If the configuration has changed, you will see the difference as a diff.
...
  -                "rev": "akanavin/sstate-for-all"
  +                "rev": "akanavin/bitbake-setup-testing"
...

If the configuration has not changed, but layer revisions referred to it have (for example
if the configuration specifies a tip of a branch), you will see that too:

===
...
Layer repository git://git.yoctoproject.org/poky-contrib checked out into /home/alex/builds/poky-alex/layers/poky updated revision akanavin/sstate-for-all from 6b842ba55f996b27c900e3de78ceac8cb3b1c492 to aeb73e29379fe6007a8adc8d94c1ac18a93e68de
===

4. If the configuration has changed, you can bring it in sync with:

$ bin/bitbake-setup update

Note that it will also rename/preserve the existing build/conf directory, and print changes
in bitbake configuration (diff of content of build/conf/) if that has changed. I can't
at the moment think of anything more clever that is also not much more brittle or complex
to implement, but open to suggestions.

Terminology
===========

- 'top directory' means the place under which bitbake-setup reads and
writes everything. bitbake-setup makes a promise to not touch anything outside of
that, unless otherwise directed to by entries in settings (currently
there is one such setting for fetcher downloads for layers and config
registries). Top directory can be selected by an environment variable, a command line option,
or otherwise assumed to be ~/bitbake-builds/. If BBPATH is in environment
(e.g. we are in a bitbake environment), then the top directory is
deduced from that and doesn't need to be specified by hand.

- 'settings' means bitbake-setup operational parameters that are
global to all builds under a top directory. E.g. the location of
configuration registry, or where the bitbake fetcher should place the
downloads (DL_DIR setting). Settings are stored in a .conf file in ini
format just under the top directory.

- 'build' means a tree structure set up by 'bitbake-setup init',
consisting of, at least, a layers checkout, and a bitbake
build. It maps 1:1 to the json data it was constructed from, which is
called 'build configuration'. Build configurations are constructed from
generic configurations that may involve making one or more choices
about available options in them. Generic configurations are files, URLs
or are obtained from git repositories called 'config
registries', in which case they can be listed with 'bitbake-setup
list'. There can be multiple 'builds' under a top directory. Here are
two example generic configurations that showcase this:
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-options.conf.json
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-nested-configs.conf.json

- 'bitbake-setup status' will tell if a build is in sync with
the generic configuration it was made from. 'bitbake-setup update' will bring a build
in sync with a configuration if needed.

- 'bitbake build' means a particular sub-tree inside a build that
bitbake itself operates on, e.g. what is set in BBPATH/BUILDDIR
by oe-init-build-env. conf/* in that tree is 'bitbake configuration'.
Bitbake configurations are constructed from templates and fragments,
with existing mechanisms provided by oe-core. The configuration file
format is specified such that other mechanisms to set up a
bitbake build can be added; there was a mention of ability to specify
local.conf content and a set of layers directly in a configuration. I
think that scales poorly compared to templates and fragments, but I
made sure alternative ways to configure a bitbake build are possible
to add in the future :)

- 'source override' is a json file that can be used to modify revisions
and origins of layers that need to be checkout into a build (e.g.
when master branches need to be changed to master-next for purposes
of testing). Such a file is specified with a command-line option to 'init'
and an example can be seen here:
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-next.override.json

This commit includes fixes by
Ryan Eatmon <reatmon@ti.com> https://github.com/kanavin/bitbake/pull/1
Gyorgy Sarvari <skandigraun@gmail.com> https://github.com/kanavin/bitbake/pull/2
Johannes Schneider <johannes.schneider@leica-geosystems.com> https://github.com/kanavin/bitbake/pull/3 https://github.com/kanavin/bitbake/pull/5

(Bitbake rev: b96154aeb1fc89184ac245e0d68e6e726fe80c04)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-14 11:24:57 +01:00
Peter Kjellerstedt
4a1f38dd8d bitbake: knotty, uihelper: Remove running_pids and lastpids
* lastpids has not been used for almost 10 years.
* There is no longer any need to use running_pids to keep track of the
  order the pids in running_tasks were added as dicts are guaranteed to
  remember the insertion order since Python 3.7.

(Bitbake rev: 562b5bf0be2883144391e7030a9dce6a233802ac)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:30:31 +01:00
Yoann Congal
b203f9146e bitbake: ast: Warn on multiple builtin config fragments for the same variable
Having multiple builtin config fragments for the same variable
(eg OE_FRAGMENTS = "... machine/A ... machine/B") is not supported.
Warn the user to make them fix this but continue with the normal
variable evaluation : the last affectation "wins".

Added warning looks like:
WARNING: Multiple builtin fragments are enabled for machine via variable OE_FRAGMENTS: machine/qemux86-64 machine/test machine/qemux86-64. This likely points to a mis-configuration in the metadata, as only one of them should be set. The build will use the last value.

(Bitbake rev: 1c12aa23f6678dc289fc0e0d8b4dad311bd39c35)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:30:31 +01:00
Mikko Rapeli
34f0052af3 parselogs-ignores-genericarm64.txt: add Qualcomm rb3gen2 workarounds
u-boot 2025.10rc2 based firmware from meta-ts
https://gitlab.com/Linaro/trustedsubstrate/meta-ts
triggers some non-fatal errors from various drivers.
Ignore them for now since all other oeqa runtime tests pass
since in genericarm64 we don't build this firmware.

(From meta-yocto rev: fe65e142d0d9ba0e51ff9175ffa82e902f982a20)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
bdf976a085 genericarm64.conf: increase initramfs size
With firmware files and more kernel modules the size
limit is hit again. Increase by 60Mb.

(From meta-yocto rev: 4348f496cf4950cf665d7c56643718b5238e96bf)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
657786eebd genericarm64.conf: install Qualcomm rb3gen2 firmware
Copied from meta-qcom
recipes-bsp/packagegroups/packagegroup-rb3gen2.bb
Sadly does not fully resolve kernel error messages
from WiFi firmware files.

(From meta-yocto rev: 6fd803cd96746cbf3ae625d0fadbb176ce1b2919)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
52149efc8c parselogs-ignores-genericarm64.txt: add Rpi4 wifi firmware workaround
The firmware file is not available from linux-firmware package
so it may be that another file name is used instead. Thus
this non-fatal error can be ignored for now.

(From meta-yocto rev: 2d442f338cbd4a681388c093899a5709b7881d96)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
13e1a2b2c7 genericarm64.conf: install linux-firmware-bcm43455 to images
by default. Fixes parselogs oeqa runtime test failures:

https://ledge.validation.linaro.org/scheduler/job/121516

Central error: brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin failed with error -2
...
Central error: brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.bin failed with error -2

(From meta-yocto rev: dce46431683b5172327770972166c79e4c60cc2d)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
3e48365f40 parselogs-ignores-genericarm64.txt: add workarounds for NXP imx8mp
u-boot 2025.10rc2 based firmware from meta-ts
https://gitlab.com/Linaro/trustedsubstrate/meta-ts
triggers some non-fatal errors from various drivers.
Ignore them for now since all other oeqa runtime tests pass
since in genericarm64 we don't build this firmware.

(From meta-yocto rev: 0b15a104ea04acd79b87f7cf3da96f637dafdef4)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
1f92e7fd5d parselogs-ignores-genericarm64.txt: add workaround for TI K3 am62p5
With u-boot 2024.04 based firmware the board firmware seems to
have one small issue which kernel complains about. The board
works well and passes other tests so ignore this for now.

(From meta-yocto rev: 56f33867249f79340f868cb90bc7e4c2a6a641d2)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
a39acbdcb7 parselogs-ignores-genericarm64.txt: add workarounds for AMD zcu102
u-boot 2025.10rc2 based firmware from meta-ts
https://gitlab.com/Linaro/trustedsubstrate/meta-ts
triggers some non-fatal errors from various drivers.
Ignore them for now since all other oeqa runtime tests pass
since in genericarm64 we don't build this firmware.

(From meta-yocto rev: 6fc9c8e2a8b23877c2cd2bac43d37dcaf5b6c8ef)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:52 +01:00
Mikko Rapeli
4f18298d25 parselogs-ignores-genericarm64.txt: add error messages from rockpi4b
Rockchip rk3399 based rockpi4b with u-boot 2025.10rc2 based firmware
from meta-ts https://gitlab.com/Linaro/trustedsubstrate/meta-ts
triggers some non-fatal errors from various drivers.
Ignore them for now since all other oeqa runtime tests pass
since in genericarm64 we don't build this firmware.

https://ledge.validation.linaro.org/scheduler/job/119422

(From meta-yocto rev: b91b6ca6827d489ecb21cb688403c2b1c76ccd52)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 23:29:51 +01:00
Mathieu Dubois-Briand
22d697a835 oeqa: runtime: logrotate: Fix typos and add comments
(From OE-Core rev: 7d1d3b4958bc48cf49ddb653c76c8cd536babac6)

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Dmitry Baryshkov
2d844e061a x264: switch to PACKAGECONFIG
Switch to PACKAGECONFIG in order to make x264 options configurable.
Enable FFmpeg (by default) and OpenCL (if enabled by the distro). Pick
up the patch to fix building with the latest FFmpeg.

(From OE-Core rev: 4339bf44faa11bf5933f23ac5b0b6ecaa5a1afab)

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Gyorgy Sarvari
f897ff7da3 vte: upgrade 0.80.3 -> 0.82.1
Commitlog:
https://gitlab.gnome.org/GNOME/vte/-/compare/0.80.3...0.82.1

(From OE-Core rev: d2e5df7e3daa4b58d8802b62fbbba5f6959379c3)

Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Khem Raj
fa64301121 llvm/clang: Upgrade to 21.1.3 release
brings following fixes

* 450f52eec88f Bump version to 21.1.3.
* 05b5090e961f Port 5b4819e to release/21.x
* 2cb08dbb39b1 [compiler-rt][sanitizer] fix msghdr for musl (#136195)
* 6e687cbe0dd3 [SPARC] Prevent meta instructions from being inserted into delay slots (#161111)
* f8151a1d2c86 [clang] [Headers] Don't use unreserved names in avx10_2bf16intrin.h (#161824)
* 9ee4ac8a8359 [clang][SPARC] Pass 16-aligned structs with the correct alignment in CC (#155829)
* 220bac16a417 [Hexagon] Add opcode V6_vS32Ub_npred_ai for offset validity check (#161618)
* a867bd53e861 [clang][PAC] Don't try to diagnose use of pointer auth on dependent types #159505 (#159859)
* 41e817a1d1f4 release/21.x: [clang-format] Fix bugs in annotating arrows and square brackets (#160973)
* 559d966bcb54 [Mips] Fix atomic min/max generate mips4 instructions when compiling for mips2 (#159717)
* d1e2f8916128 [LLD] [COFF] Fix symbol names for import thunks (#160694)
* 0060034c6a0b [analyzer] Revert #115918, so empty base class optimization works again (#157480)
* 31e4363ba9c2 [NVPTX] Disable relative lookup tables (#159748)
* 74cb34a6f51a Bump version to 21.1.3

(From OE-Core rev: 4a590bdd8794c6873f9ff0dc6d988d62f5b2ca5c)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Dmitry Baryshkov
1eadde0f93 vulkan: upgrade 1.4.321.0 -> 1.4.328.1
Upgrade all Vulkan-related packages together in a single commit.

License-Update: glslang dropped SPIR-V remapper
(From OE-Core rev: 55c7566c33833a8f36cbf50a70cb1a1cf7aa96d5)

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Yi Zhao
27c46b89fe ltp: upgrade 20250530 -> 20250930
ChangeLog:
https://github.com/linux-test-project/ltp/releases/tag/20250930

(From OE-Core rev: dd53cccdd11358398cf154ee8d0e529ff180adb1)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Yi Zhao
739fb7e209 json-glib: upgrade 1.10.6 -> 1.10.8
ChangeLog:
 * Add missing escape
 * Fix static builds

(From OE-Core rev: b73e30c82d16e2a93e402873cad0da6ca6251f2b)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Yoann Congal
a41bfa10a0 oeqa/selftest/buildoptions: Fix test_yocto_source_mirror MIRRORS & PREMIRRORS definitions
Currently, the definitions of MIRRORS and PREMIRRORS made by
test_yocto_source_mirror() are overwritten when run on the autobuilder:
* MIRRORS = "" is extended by mirrors.bbclass to its usual value.
* PREMIRRORS = "* dl.yp.org" is overwritten by the autobuilder.conf
  config fragment to ""

Simplified variable history:
  # $MIRRORS [4 operations]
  #   set .../contribution-yocto/builds/build-qemux86-64-st/conf/selftest.inc:3
  #     ""
  #   append .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/classes-global/mirrors.bbclass:57
  #     "${DEBIAN_MIRROR}	..."
  MIRRORS=" http://deb.debian.org/debian/pool ..."

  # $PREMIRRORS [4 operations]
  #   set .../contribution-yocto/builds/build-qemux86-64-st/conf/selftest.inc:18
  #     "    bzr://.*/.*   http://downloads.yoctoproject.org/mirror/sources/ \n ..."
  #   set .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/conf/fragments/yocto-autobuilder/autobuilder.conf:6
  #     ""
  #   append .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/classes-global/mirrors.bbclass:84
  #     "git://sourceware.org/git/glibc.git https://downloads.yoctoproject.org/mirror/sources/  ..."
  PREMIRRORS=" git://sourceware.org/git/glibc.git https://downloads.yoctoproject.org/mirror/sources/  ..."

Despite this unexpected configuration, the test passes because after
failing to find a PREMIRROR (empty), failing to download the normal
SRC_URI (because of BB_ALLOWED_NETWORKS), the fetcher tries the MIRRORS
which have by default download.yoctoproject.org. For example, on a
failed meta-oe-mirror test[0], we can see a lot of warnings:
"stdio: WARNING: ... do_fetch: Failed to fetch URL https://... attempting MIRRORS if available"

By using the ":forcevariable" override, test_yocto_source_mirror() makes
sure the correct value is set for MIRRORS and PREMIRRORS (whatever is
the configuration).

Simplified variable history (after the fix):
  # $MIRRORS [4 operations]
  #   append .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/classes-global/mirrors.bbclass:57
  #     "${DEBIAN_MIRROR}	..."
  #   append .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/classes-global/mirrors.bbclass:68
  #     "git://salsa.debian.org/.* ..."
  #   override[forcevariable]:set .../contribution-yocto/builds/build-qemux86-64-st/conf/selftest.inc:3
  #     ""
  MIRRORS=""

  # $PREMIRRORS [4 operations]
  #   set .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/conf/fragments/yocto-autobuilder/autobuilder.conf:6
  #     ""
  #   append .../contribution-yocto/builds/build-qemux86-64-st/../../layers/poky/meta/classes-global/mirrors.bbclass:84
  #     "git://sourceware.org/git/glibc.git ..."
  #   override[forcevariable]:set .../contribution-yocto/builds/build-qemux86-64-st/conf/selftest.inc:18
  #     "    bzr://.*/.*   http://downloads.yoctoproject.org/mirror/sources/ ..."
  # pre-expansion value:
  #   "    bzr://.*/.*   http://downloads.yoctoproject.org/mirror/sources/ ..."
  PREMIRRORS="    bzr://.*/.*   http://downloads.yoctoproject.org/mirror/sources/ ..."

[0]: https://autobuilder.yoctoproject.org/valkyrie/#/builders/82/builds/470/steps/17/logs/warnings

Fixes [YOCTO #15993]

(From OE-Core rev: ab810d1239d4db0bc0f23db31c70cc9c6e59357e)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00
Dmitry Baryshkov
c4c27366f5 mesa: sort out driver lists
Sort the lists of VUKAN_DRIVERS, GALLIUMDRIVERS and driver-related
PACAKGECONFIG entries, making it easier to add new drivers or to find
out the correct dependencies for the driver.

(From OE-Core rev: 64108679ada885b74fcc6ac49f9e5e983f34e663)

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-10-13 18:01:04 +01:00