This adds support for the new bitbake-setup:
* Maintain backwards compatibility for poky-classic releases
* Assume bitbake-setup for all releases 'master' and forward
* When no ".templateconf" assume Poky and not OE
* Generate and use "init-build-env" script instead of 'oe-init-build-env'
directly, but "cd" to the clone and not the local installation.
* Include "bblayers.conf" because "cookerdata.py" now explicitly
required it, and let it be over-ridden by the usual "toaster-bblayers.conf".
* Force the value "TOPDIR" in "toaster.conf" to the local project's directory.
Normally "cookerdata.py" sets TOPDIR from getcwd(), but somehow in bitbake-setup
mode the CWD at this point has become ".../layers/openembedded-core", so
"bitbake.conf and all other files cannot be found. This non-obvious problem was
hard to discover. Until this is understood, the forced value will make things work.
[YOCTO #16012]
(Bitbake rev: 22b3fd418efc10da7e4a8c1725de285714d2396b)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After a terminology review by Antonin we brainstormed
something less confusing than 'build' and 'build directory'
for the place where bitbake-setup clones layers and
creates a *bitbake* build directory in. People are bound
to get these two confused and mix them up, and 'setup' is
much more distinct and aligns nicely with 'bitbake-setup'.
It's also not claimed by anything else in OE/Yocto.
So before:
top-dir -> build-dir (can be several) -> bitbake build dir, layer dir, config dir
Now:
top-dir -> setup-dir (can be several) -> bitbake build dir, layer dir, config dir
This also updates the respective command line options, I understand it's
a breaking change, but as before the tweaks are simple and we need to get the
terminology right for the users, and now is the time to do it.
(Bitbake rev: eeb81a35bf0304451f7612950d5156ea7ff18bad)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the sub-command 'settings' would take any number of
arguments and then silently do nothing if the number wasn't three.
The help text was also not clear about this, marking the positionals
separately as optional:
usage: bitbake-setup settings [-h] [--global] [--unset UNSET UNSET] [-l] [section] [key] [value]
The '--unset SECTION SETTING' also did not integrate too well, as it
had its own positional arguments for section+setting.
For a bit more consistency and a explorable help, a sub-subparser is
added, that provides the commands:
bitbake-setup settings list
bitbake-setup settings set foo bar baz
bitbake-setup settings unset foo bar
with a '--global' that is added from a stand-alone parent parser, so
that it shows up in all sub-command help texts.
The new help text now reads:
usage: bitbake-setup settings [-h] [--global] {list,set,unset} ...
and the respective sub commands:
usage: bitbake-setup settings list [-h] [--global]
usage: bitbake-setup settings set [-h] [--global] <section> <setting> <value>
usage: bitbake-setup settings unset [-h] [--global] <section> <setting>
(Bitbake rev: 8b582ef8dd0cef0192d4c0104bcd9b5d642d132c)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After some further feedback, additional changes are made:
1. 'setting' command is renamed to 'settings' to better reflect
that it is an interface to various ways of managing settings.
2. This command now has a -l/--list option to list all settings
with their values (same as 'git config -l').
3. A new level of settings (built-in defaults) is added,
and used as a last resort after command line options, top dir
settings file and global settings file.
4. This means bitbake-setup does not have to write and use a
global settings file, and it no longer does so when initializing
a build, avoiding default 'pollution' of ~/.config/bitbake-setup/
which can be problematic or unwelcome.
A global settings file is still created if a setting is explicitly
requested to be placed into it.
5. 'install-global-settins' is removed as the use case for it
(tweak default settings before using them to initialize a build)
can be achieved by setting the settings individually.
5. Similarly, a top dir settings file is no longer created by default
and only appears if a setting needs to be written into it.
6. Default dl-dir is again created inside a top directory and not
in ~/.cache/ to make default builds fully contained in the top
directory (which was also asked about).
(Bitbake rev: 664f8ec48d42d2ddc5f234c4f7d590fa597f489a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the outcome of various discussions, suggestions and pull
requests on github.
What has specifically changed?
1. The sources for the settings are no longer separated, but are stacked and given priorities,
from highest to lowest:
a. '--setting section key value' on the command line
b. a settings file in the top directory
c. a global settings file in ~/.config/bitbake-setup/ (or in a file pointed to by --global-settings)
Any setting can be in any of these three locations (other than top dir name and prefix which do not
make sense in the settings file in the top directory).
2. A global settings file must contain all of the needed settings, while a settings file
in the top directory can be empty (and this is how they are written out if they do not exist).
Specifically, both dl-dir and registry settings have been relocated to the global file,
and dl-dir defaults to ~/.cache/bitbake-setup/downloads, rather than somewhere in top dir.
3. The file name for both global and top dir settings is now 'settings.conf'.
4. --top-dir-prefix and --top-dir-name options have been removed and superseded by
a generic, universal --setting option.
5. 'install-settings' command has been removed, as it is no longer does anything useful,
and is superseded by the 'setting' command (see below).
'install-global-settings' has been retained, to be able to have a set of global defaults
that can be changed without initializing a build.
6. 'change-setting', 'change-global-setting' and 'install-settings' have all been replaced
by a single 'setting' command that mimics 'git config' in its parameters:
a. Changing a setting: bitbake-setup setting [--global] default dl-dir /path/to/downloads
b. Removing a setting: bitbake-setup setting [--global] --unset default dl-dir
(Bitbake rev: 713e7f213c6d4a620be9ce34d5f4396af48e1d69)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test configuration to cover the 'bb-env-passthrough-additions'
conf.json key, and add it to the test routine.
(Bitbake rev: 24f12b68692f9ebb5d3813bc3b1771e43298f640)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-setup conf.json can be purely template driven, to setup older
yocto LTS (e.g. scarthgap) based layer-collections - where fragment
support was not yet present in oe-core+bitbake.
Add a configuration for that scenario and add it to the test routine.
(Bitbake rev: 23e121befa0779fbb1f342984b583c04ccc637a0)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This implement the ability to use "{THISDIR}/my-layer" in the
"bb-layers" list. "{THISDIR}" is remplaced by the directory containing
the configuration file.
In small projects, we try to keep the setup a simple as possible: a
single git repo containing both the build confguration (e.g.
a bitbake-setup configuration file) and the meta layer with project
recipes/machine/distro.
This change allows this kind of setup:
├── meta-my-project/ # the project layer
└── my-project.conf.json # the bb-setup configuration file
by writing, in my-project.conf.json:
"bitbake-setup": {
"configurations": [{
"bb-layers": [
"{THISDIR}/meta-my-project"
Note: in this case meta-my-project is not present as a "source", so, not
handled by bb-setup update/status. It is expected of the user to handle
this on their own (is our case, a simple git workflow).
(Bitbake rev: b3153be29de8b8570b0c184369bd41f4c646cf92)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-setup deduces top directory from BBPATH, which, if set, interferes with
the tests' own setup.
(Bitbake rev: e974d42eb5229f755e14b46a00ad06b23b53e143)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes maintaining and extending them easier.
(Bitbake rev: 16dc8e3dad7dde7e7651cce13549e61574cafba1)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously bitbake-setup was checking out 'detached commits' using
fetcher's nobranch feature, as that is the only option when only a revision is in the config.
Branches are optional, but beneficial, as
- checkout directory will be on a branch, making it easier for users
to understand where they are if they need to make changes (also
bitbake will print branch information instead of saying 'HEAD:sha').
- supply chain security! Enforcing a branch means any specified revision
has to be on it, and no one can sneak in (accidentally or deliberately!)
some dangling commit, or something from their private branch in the same repo.
(Bitbake rev: 45ed9b9faebdaa8cb7cc8dd2a6d51ec8eea06e73)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This basically calls install-buildtools from oe-core/poky, but
it ensures via command line parameters that the installation
location is stable and the downloads are preserved for reproducibility:
$ bin/bitbake-setup install-buildtools
Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf
======
Buildtools archive is downloaded into /home/alex/bitbake-builds/yocto-master-testing/buildtools-downloads/20250319141333 and its content installed into /home/alex/bitbake-builds/yocto-master-testing/buildtools
... (output from install-buildtools script)
======
It also detects when buildtools are already installed, and will direct
users what to do:
======
alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup install-buildtools
Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf
Buildtools are already installed in /home/alex/bitbake-builds/yocto-master-testing/buildtools.
If you wish to use them, you need to source the the environment setup script e.g.
$ . /home/alex/bitbake-builds/yocto-master-testing/buildtools/environment-setup-x86_64-pokysdk-linux
You can also re-run bitbake-setup install-buildtools with --force option to force a reinstallation
======
This commits includes fixes by Gyorgy Sarvari <skandigraun@gmail.com>
https://github.com/kanavin/bitbake/pull/2
(Bitbake rev: 3fe3096847046110c72b23fce37fb4a459b1d748)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run like this:
alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-selftest -v bb.tests.setup
test_setup (bb.tests.setup.BitbakeSetupTest.test_setup) ... ok
----------------------------------------------------------------------
Ran 1 test in 9.223s
OK
The test does a basic run-through of init, then status/update
on an unchanged configuration, then status/update on a
configuration changed via new commits to the test layer,
then status/update on configuration changed via the top
level json config file.
Note that nothing whatsoever is fetched from the network;
the test relies entirely on synthetic data contained inside
itself, including minimal stubs for oe-setup-build and
bitbake-config-build. This data is used to create temporary
git repositories then clone them via local filesystem URIs.
Later on this can be supplemented by an oe-selftest that
tests bitbake-setup against real config files in the
official configuration repository and real layers,
templates and fragments.
(Bitbake rev: e3aa3eb46bd3196fa5415fa36e3737636fd6a1c0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lastpids has not been used for almost 10 years.
* There is no longer any need to use running_pids to keep track of the
order the pids in running_tasks were added as dicts are guaranteed to
remember the insertion order since Python 3.7.
(Bitbake rev: 562b5bf0be2883144391e7030a9dce6a233802ac)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having multiple builtin config fragments for the same variable
(eg OE_FRAGMENTS = "... machine/A ... machine/B") is not supported.
Warn the user to make them fix this but continue with the normal
variable evaluation : the last affectation "wins".
Added warning looks like:
WARNING: Multiple builtin fragments are enabled for machine via variable OE_FRAGMENTS: machine/qemux86-64 machine/test machine/qemux86-64. This likely points to a mis-configuration in the metadata, as only one of them should be set. The build will use the last value.
(Bitbake rev: 1c12aa23f6678dc289fc0e0d8b4dad311bd39c35)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Imagine a machine fragment machine/A and a configuration which sets:
MACHINE = "B"
MACHINE:forcevariable = "C"
As I understand it, the fragment behaviour was intended to replace the
MACHINE = "B", so the override would still be active. The current code
replaces all variable overrides.
parsing=True, switches to the other behaviour, which I believe was the
design intent and the behaviour users would expect.
This is useful to allow test configurations to override a MACHINE setting
without change the fragments which would complicate the test code.
(Bitbake rev: f65bc6aaf4c11bc7e566c895209c093627a3015b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simple tests for various ways to include and require configuration
files.
(Bitbake rev: bdcb67824172ed5cd76fa0274dc3d27aeb52e77c)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no reason to call file.flush() as the last call within a file
context manager since ending the context will close the file and thus
flush it.
(Bitbake rev: 90d5ce926d434d60d6df28b7d77b3cbc9b62ce14)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tags for bitbake 2.8.6 and 2.8.7 have been removed from the git: use
some other ones.
(Bitbake rev: dcf12947954f3184d99fbf9813b3f3f667dacc5f)
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case a recipe specifies a git SRC_URI along with revision and tag, but only the
revision is present in the local clone without the tag (because it was tagged after
it was cloned), then unpacking fails with the following error:
... rev-list -n 1 1.0 failed with exit code 128, output:\nfatal: ambiguous argument \'1.0\': unknown revision or path not in the working tree
This happens because the during the download step only the revision's presence is
verified to decide if the repository needs to be updated.
To avoid this, check also if the tag is present in the local repository, when the "tag"
tag is specified.
(Bitbake rev: 546b347b4d3d82c01ecc99f45296f66e44638adc)
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactoring prints, print() functions were eliminated from all loops and it uses
"\n".join(...) in a single print() call for better performance.
(Bitbake rev: c32c3d9b83818661e12f3e437563ab4e1fa05e15)
Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refresh footer in 40 Hz to avoid heavy print() flooding but keep it fluent for human eyes.
(Bitbake rev: c36efdf642d858c6997819744d00a3c1965c6417)
Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optimize printing in footer update with use a StringIO buffer and it
prints content to terminal in a single call in every cycle.
(Bitbake rev: 32ba622d78f20b231f30f848379b4bbc3d7414da)
Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Introduce self._fd_console as a dedicated attribute of self._handle_resize().
(Bitbake rev: f8c76eb89d52b1c28407f0b52dfe4318faa47cd2)
Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prior to commit aa84a900e ("cooker: Ensure delays are accurately transfered to
the idle loops from runqueue"), this was necessary. But now retval is returned
directly.
(Bitbake rev: c2eb4336fe10b1bf8bbc6291c32ca362840f39e1)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Marks bb.utils.explode_deps as callable from filter functions
(Bitbake rev: bb07003641e76de994482f7835a432f20297af96)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a new setVarFilter() API to the data store allowing filters to be
applied to variables.
Note that filters are applied to the non-override part of the variable name
so a filter set against RDEPENDS would apply against RDEPENDS:${PN} and
friends.
The filter function is applied before returning the final variable value.
(Bitbake rev: a9471c10d1de039474ddb4738abd286b928d82f4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the python API for applying filters to a string and being able to
register functions as filters.
Filter functions are pure functions where an input is translated into
an output and there are no external data accesses. This means translations
can be cached as they won't change.
(Bitbake rev: 7d25d7511ca14213eea78ee739d260295cfa4045)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parsing failures occur, trap KeyError to avoid these kinds
of tracebacks (from a parsing error in meta-ti).
bb.parse.ParseError: ParseError at /srv/pokybuild/yocto-worker/check-layer-nightly/build/meta-ti/meta-ti-bsp/recipes-kernel/linux/linux-ti-mainline_git.bb:6: Could not inherit file classes/${KERNEL_BASE_CLASS}.bbclass
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/command.py", line 123, in runAsyncCommand
self.cooker.updateCache()
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 1639, in updateCache
if not self.parser.parse_next():
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2314, in parse_next
self.shutdown(clean=False, eventmsg=str(exc))
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2209, in shutdown
read_results()
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cooker.py", line 2195, in read_results
self.result_queue.get(timeout=0.25)
File "/usr/lib/python3.10/multiprocessing/queues.py", line 122, in get
return _ForkingPickler.loads(res)
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cache.py", line 338, in __setstate__
setattr(self, key, self._restore(state[key], pid))
File "/srv/pokybuild/yocto-worker/check-layer-nightly/build/lib/bb/cache.py", line 318, in _restore
ret[dep] = map[mapnum]
KeyError: 156
(Bitbake rev: 750c68ee7ee3f3d4518348e4c948243504880770)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO 15917]
When fetching a git repository the .gitattributes file is scanned, checking if LFS
support is required for the repository. This scan checks if the word "lfs" is present
in the file, however the used regex doesn't account for comments, which makes some
repositories[1] be to misidentified as requiring LFS support (which fails fetching, in case
lfs support isn't installed on the build host).
To avoid it, change the used regex to ignore lines starting with "#".
[1]: e.g. https://github.com/MicrochipTech/cryptoauthlib
(Bitbake rev: 7917a758fc328747116c7899e689171bd0efc883)
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When recreating the uri in wget's checkstatus method, we only use the
scheme, netloc and path. This completely strips the query parameters
from the final URI and potentially breaks the checking functionality
from certain fetchers.
This is the case for the Azure storage fetcher, as it requires a SAS
token that is formatted as a series of query parameters. The error
manifests itself when using a private storage account as a PREMIRROR or
SSTATE_MIRROR (since regular SRC_URI won't run the checkstatus).
This problem is present in scarthgap, but wasn't in kirkstone.
CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
(Bitbake rev: 096301250455e2a83bdd818a56317c62436c9981)
Signed-off-by: Philippe-Alexandre Mathieu <pamathieu@poum.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This API is no longer used and bitbake has moved beyond Python 2.7.3 as
the minimum version, so remove it.
(Bitbake rev: 0eb7b5dd512ed8d8b77b5779858b9fbd99edb4a4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3.14 changes the default multiprocessing context from "fork" to
"forkserver"; however bitbake heavily relies on "fork" to efficiently
pass data to the child processes. As such, make "fork" context in the bb
namespace and use it in place of the normal multiprocessing module.
Note that multiprocessing contexts were added in Python 3.4, so this
should be safe to use even before Python 3.14
[YOCTO #15858]
(Bitbake rev: 62be9113d98fccb347c6aa0a10d5c4ee2857f8b6)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The upstream servers are having issues so switch to our own shadow copy
of the repo.
(Bitbake rev: e910c7cd24fd366d6756641cd599c4efeb492e2a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a more logical name for the profile reports and put each report
into a separate file since people struggle to discover them currently.
(Bitbake rev: a8145c84e0899285a5e6a809f1515118b002b106)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running "time bitbake -pP idle" with a valid cache shows around 800,000
calls to enum creation from python's signal.py. We don't care about this
overhead and it adversely affects cache load time quite badly.
Try and use _signal directly, falling back to signal, which avoids
this overhead we don't need and makes cache loading much faster.
(Bitbake rev: ee5fce67ce35b025c68aa61e2e758903269ee346)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Starting with python 3.12, profiling now stays enabled over threads yet
you can't extract the profile data in the threads themselves, which makes it
difficult to use for our use case.
Our main loop starts the idle loop which starts the parsing threads and this
means we can't profile in the main loop and the parsing threads or the idle
loop at the same time due to this.
Add options to the commandline so you can specify which piece of bitbake
you want to enable profiling for. This allows some profiling with python 3.12
onwards rather than crashing.
(Bitbake rev: 09f29a4968841ee5070f70277ba8c253bb14f017)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible for multple bitbake threads to empty ui_queue in parallel
leading to duplicate console messages and much confusion when debuging.
Use the lock to extract the queue data which means only one thread will
processing, removing the duplicate out of order messages.
(Bitbake rev: 945095602e40d54efb8de494218f4a2b25c9969f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have code duplication in the way we handle profiling of code sections.
Create a common function in utils which covers this.
The main loop and idle loop profile files were also reversed. Fix this and the naming,
removing a couple of unused variables containing the profile log names in the process too.
(Bitbake rev: b4f6bae97ac9607420fc49fd4c9e957d89c9a5f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the server is quickly stopped, we see tracebacks in the locks
due to the file not existing. Hide these as they're not errors.
(Bitbake rev: a7e1a07e9ef7e6f6a1bcaf567d7916a8ee1ef087)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test behavior did not change visibly though.
"bitbake-selftest bb.tests.runqueue" passes completely, just like before.
(Bitbake rev: 1751aed08f8472f20fcfbadbb09d35f951904952)
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of pre-partitioning which jobs will go to which parser
processes, pass the list of all jobs to all the parser processes
(efficiently via fork()), then used a shared counter of the next index
in the list that needs to be processed. This allows the parser processes
to run independently of needing to be feed by the parent process, and
load balances them much better.
(Bitbake rev: 373c4ddaf0e8128cc4f7d47aefa9860bd477a00f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If parsing ends early and unexpectedly, add some internal values
to better understand why/how it failed.
(Bitbake rev: 775f9720a17c9f3d6815d42c733ab5aaaa53749c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
