poky/bitbake
Richard Purdie 08aa69a1fd bitbake: runqueue: Add support for BB_LOADFACTOR_MAX
Some ditros don't enable /proc/pressure and it tends to be those which we
see bitbake timeout issues on, seemingly as load gets too high and the bitbake
processes don't get scheduled in for minutes at a time.

Add support for stopping running extra tasks if the system load average goes
above a certain threshold by setting BB_LOADFACTOR_MAX.

The value used is scaled by CPU number, so a value of 1 would be when
the load average equals the number of cpu cores of the system, under one
only starts tasks when the load average is below the number of cores.

This means you can centrally set a value such as 1.5 which will then
scale correctly to different sized machines with differing numbers
of CPUs.

The pressure regulation is probably more accurate and responsive, however
our graphs do show singificant load spikes on some workers and this
patch is aimed at trying to avoid those.

Pressure regulation is used where available in preference to this load
factor regulation when both are set.

(Bitbake rev: 14a27306f6dceb4999c2804ccae5a09cc3d8dd49)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-23 14:34:05 +00:00
..
bin bitbake: git-make-shallow: support git's safe.bareRepository 2024-02-19 12:00:59 +00:00
contrib bitbake: contrib/vim: Syntax improvements 2023-12-23 16:20:25 +00:00
doc bitbake: doc: bitbake-user-manual: remove reference to SSTATE_MIRRORS variable 2023-09-05 08:13:13 +01:00
lib bitbake: runqueue: Add support for BB_LOADFACTOR_MAX 2024-02-23 14:34:05 +00:00
.gitattributes
AUTHORS
ChangeLog
LICENSE bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.GPL-2.0-only bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.MIT bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
README bitbake: doc: README: simpler link to contributor guide 2024-02-10 14:13:51 +00:00
SECURITY.md bitbake: SECURITY.md: add file 2023-10-24 12:49:56 +01:00
toaster-requirements.txt bitbake: Update toaster-requirements to add django-log-viewer==1.1.7 2023-10-15 09:12:05 +01:00

Bitbake
=======

BitBake is a generic task execution engine that allows shell and Python tasks to be run
efficiently and in parallel while working within complex inter-task dependency constraints.
One of BitBake's main users, OpenEmbedded, takes this core and builds embedded Linux software
stacks using a task-oriented approach.

For information about Bitbake, see the OpenEmbedded website:
    https://www.openembedded.org/

Bitbake plain documentation can be found under the doc directory or its integrated
html version at the Yocto Project website:
    https://docs.yoctoproject.org

Bitbake requires Python version 3.8 or newer.

Contributing
------------

Please refer to our contributor guide here: https://docs.yoctoproject.org/contributor-guide/
for full details on how to submit changes.

As a quick guide, patches should be sent to bitbake-devel@lists.openembedded.org
The git command to do that would be:

    git send-email -M -1 --to bitbake-devel@lists.openembedded.org

If you're sending a patch related to the BitBake manual, make sure you copy
the Yocto Project documentation mailing list:

    git send-email -M -1 --to bitbake-devel@lists.openembedded.org --cc docs@lists.yoctoproject.org

Mailing list:

    https://lists.openembedded.org/g/bitbake-devel

Source code:

    https://git.openembedded.org/bitbake/

Testing
-------

Bitbake has a testsuite located in lib/bb/tests/ whichs aim to try and prevent regressions.
You can run this with "bitbake-selftest". In particular the fetcher is well covered since
it has so many corner cases. The datastore has many tests too. Testing with the testsuite is
recommended before submitting patches, particularly to the fetcher and datastore. We also
appreciate new test cases and may require them for more obscure issues.

To run the tests "zstd" and "git" must be installed.

The assumption is made that this testsuite is run from an initialized OpenEmbedded build
environment (i.e. `source oe-init-build-env` is used). If this is not the case, run the
testsuite as follows:

    export PATH=$(pwd)/bin:$PATH
    bin/bitbake-selftest

The testsuite can alternatively be executed using pytest, e.g. obtained from PyPI (in this
case, the PATH is configured automatically):

    pytest