Commit Graph

3475 Commits

Author SHA1 Message Date
Alexander Kanavin
bb2ce4dcf6 scripts/runqemu: allocate unfsd ports in a way that doesn't race or clash with unrelated processes
There is already a neat check_free_port() function for finding an available port
atomically, so use that and make two additional tweaks:

- no need to allocate two separate ports; per unfsd documentation they can be the same

- move lockfile release until after unfsd has been shut down and the port(s) used has been freed

[YOCTO #15077]

(From OE-Core rev: 343510b33650c88367f95e8d8322fae92ae901ca)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dee96e82fb04ea99ecd6c25513c7bd368df3bd37)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12 05:11:38 -10:00
Alexander Kanavin
e69c311ed6 scripts/runqemu: split lock dir creation into a reusable function
(From OE-Core rev: 2ada5f426e71e3873ba8c47dd925d8cfc103524b)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 004d6bcb067ecf1d796801fa43a98820c4efd3c7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12 05:11:38 -10:00
Frieder Paape
b954f15d66 image_types: Fix reproducible builds for initramfs and UKI img
I've encountered issues reproducing initramfs and UKI image builds,
which will be fixed with this patch.

1. initramfs
There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation.
The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes.

2. Unified Kernel Image
'--preserve-dates' is required for a static 'Time/Date' entry.
I've added '--enable-deterministic-archives' although in my case this
didn't change anything.

(From OE-Core rev: 0d8890f7c1fbea5036acefa3031dcd442b316725)

Signed-off-by: Frieder Paape <frieder@konvera.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fd027729bafb4e085ba0949e38e724f3a8cad102)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12 05:11:38 -10:00
Rusty Howell
c4d91873af oe-depends-dot: Handle new format for task-depends.dot
The .dot file created by `bitbake -g` changed formats a while ago, which
broke oe-depends-dot.

Also add some useful examples to the --help output.

(From OE-Core rev: e53842ea6c14ed8e97252626e3ae0d3cf4580fbc)

Signed-off-by: Rusty Howell <rustyhowell@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12 05:11:37 -10:00
Randolph Sapp
2337b81140 wic/bootimg-efi: if fixed-size is set then use that for mkdosfs
This is a bit of a compatibility issue more than anything. Some devices
get upset if the FAT file system contains less blocks than the
partition.

The fixed-size argument is currently respected by the partition creation
step but not by the file system creation step. Let's make it so the file
system respects this value as well.

(From OE-Core rev: d16301ccdfb97bf126738262eec594008c282df1)

Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-10 04:19:57 -10:00
Ross Burton
4c5a969494 scripts/lib/buildstats: handle top-level build_stats not being complete
If we try to parse a buildstats directory which was either aborted or
is still being built then the top-level build_stats file doesn't
contain an elapsed value which causes an exception:

UnboundLocalError: local variable 'elapsed' referenced before assignment

Default both start and elapsed to 0 so that the parse succeeds.

(From OE-Core rev: 19815bb76e414a1c1fd8d31a84492d72dd29205e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 701d985aa8f2e9c2b9c0736fa25b424f3701889e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-11 11:31:52 +01:00
Richard Purdie
0a1007f604 pybootchartui: Fix python syntax issue
Fix:

scripts/pybootchartgui/pybootchartgui/parsing.py:134: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if pid is 0:

(From OE-Core rev: 2429ae83400af67b1cff2adb406ec84b3147ed5e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c9a6511ae618035b8efad01646e37ba28ce1e3f8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:31:54 +01:00
Ross Burton
14b55ab826 lib/resulttool: fix typo breaking resulttool log --ptest
ptestresult_get_log() looked for a key called 'ptestresuls.sections',
which should be 'ptestresult.sections'

(From OE-Core rev: 213ed03f5a15be639a97dca6f5bbf810f7ac2eb5)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7c8c9f7283e54bf8b1521fbaad7dceb66a8fcdbb)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:31:54 +01:00
Alexander Kanavin
58d232e126 devtool/upgrade: do not delete the workspace/recipes directory
If it exists, there is no need to delete it, and if it does not,
devtool prints an ugly traceback.

(From OE-Core rev: 4ad488d2453525b7196e6d2406ac526412e3c560)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit af82e59e8f08369aabd5fa6eb43022982d4e59a7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:31:53 +01:00
Mikko Rapeli
351110f0cf runqemu: kill qemu if it hangs
qemu doesn't always behave well and can hang too.
kill it with force if it was still alive. Move clean up
commands into cleanup() function.

(From OE-Core rev: 929e7679c1d9b21ac5130a9cbc83c415fb9f63f5)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 079c2935d2f585ce49e1c7daab2155fcf0094c48)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 13:19:03 +00:00
Ross Burton
c6b64d2872 lib/buildstats: handle tasks that never finished
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.

(From OE-Core rev: ffce2744f5833d1c30db7f50443503b3c1a1d119)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23ebaec476dc46aebe5997f025661137f3e341bd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 13:19:03 +00:00
Pavel Zhukov
9a832b4628 wic: Fix usage of fstype=none in wic
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.

Example;
part /data --ondisk mmcblk0 --fstype=none  --align 4096 --fixed-size 512

will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.

(From OE-Core rev: 9af61442c8bc2e0b447706734cb90b3f4bcb2d6b)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 351cb64da37aa43113e5192605d04436652aa3b8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 13:19:02 +00:00
Martin Jansa
54d0147307 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: de7bf6689a19dc614ce4b39c84ffd825bee1b962)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 26c74fd10614582e177437608908eb43688ab510)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 24a86d0c55ee89ae0dc77975e1d0ee02898d2289)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 21:46:56 +00:00
Alexander Kanavin
0118853b5b devtool: process local files only for the main branch
devtool modify/upgrade are not currently equipped to handle conditional local files
in SRC_URI, and provide only the main no-override set in a workspace under
source/component/oe-local-files/ (this is done via meta/classes/devtool-source.bbclass).

On the other hand, updating the changes from workspace into a recipe
is run iteratively against all overrides; this works for patches (as they
all are directed into their own override branches in the workspace
git source tree), but breaks down when trying to match local files
in a workspace against local files in overridden SRC_URI lists, resulting in
bad recipe breakage.

(there's an additional twist here: existing code has a guard against this
but the guard relies on metadata in workspace .bbappend that is only there
in modify operations, but not upgrades. This commit replaces the guard
with a general check that will work everywhere).

Implementing multiple sets of local files is significant work; let's for now
simply not touch local files in recipes except when on the no-override variant.

Also, adjust the selftest cases to include conditional local files in sample
recipes, so the situation is covered by the tests.

(From OE-Core rev: 4760fac939a6204e3cb7dcd3699cd9a2508f9dee)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3a8654b860fa98f94e80c3c3fff359ffed14bbe7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-15 11:05:15 +00:00
Alexander Kanavin
45f3a4fd63 devtool/upgrade: correctly handle recipes where S is a subdir of upstream tree
'devtool modify' writes additional settings to workspace .bbappend so that this
can be handled correctly, but 'devtool upgrade' does not. This adds the missing
settings.

In particular, local files should not anymore mysteriously disappear from
SRC_URIs on upgrades.

(From OE-Core rev: 22f210f9f4dfc9e13fdc229147fd1677db5a35ec)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 0817aa5537a8d7cc9591c53dfaa1d225f4c327f7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-06 17:33:23 +00:00
Richard Purdie
5c7103d6c7 yocto-check-layer: Allow OE-Core to be tested
For unknown reasons we've never seemingly run the check layer script
against OE-Core itself. This isn't entirely straightforward as the core
layer is a bit of a special case, we can't for example compare signatures
against ourselve and we can't remove core from bblayers.conf.

Core does have distro, machine and software components too, in the case
of distro, our fallback default settings. Whilst the qemu machines could
be split into a seperate layer directory, core wouldn't then parse at all
standalone due to the lack of any machine so it seems a bit pointless to
do that.

These changes tweak the script to handle core's special cases, specifically
to allow distro and machine directories and to account for the README placed
a directory level higher than other layers.

(From OE-Core rev: 2d0641df27196ad029d95c52fb790122b2db3f9f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 23:05:50 +00:00
Ross Burton
6e1c131b2f combo-layer: add sync-revs command
When starting to use combo-layer, or if someone else is using it too,
the local last_revision may be incorrect.

This command will forcibly update the last_revision config values to the
latest SHA on the remote branch that is tracked.

(From OE-Core rev: ba24231de6a3826558c9fcd403b075fc6772fffa)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2bb5d12ecd1b0273983f7c05699f34dd64b11c25)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 23:05:50 +00:00
Ross Burton
3cdbb5cc24 combo-layer: dont use bb.utils.rename
Bitbake may not be configured, and bb isn't imported anyway.

Instead just use os.rename(), and take the filename from the file object
instead of duplicating logic.

(From OE-Core rev: 87543ed6550f7e78b3ddcfed0ef088bea932ba57)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 528f4fb3683d048537604e4562ea758968060d62)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 23:05:50 +00:00
Ross Burton
f86718661a combo-layer: remove unused import
(From OE-Core rev: 240c7dcad2f2e25efa269618c3bd6c549289207e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ebfab6c3034d41252d19c6e1a0ba79072aa51146)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 23:05:50 +00:00
Ross Burton
b627db0656 lib/buildstats: fix parsing of trees with reduced_proc_pressure directories
The /proc/pressure support in buildstats is creating directories in the
buildstats tree called reduced_proc_pressure, which confuses the parsing
logic as that cannot be parsed as a name-epoc-version-revision tuple.

Explicitly skip this directory to solve the problem.

(From OE-Core rev: 96cf8b322f4cdec31175cc2c1082e9e3bc488118)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 24f0331f0b7e51161b1fa43d4592b491d2037fe9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 23:05:50 +00:00
Joshua Watt
02f81ae7c5 scripts: convert-overrides: Allow command-line customizations
Adds argument parsing to the conversion script so that the fields that
the script uses to do conversions can be customized on the command line.
The intention is to allows easier customization without having to fork
the script, and allow automated checking on 3rd party layers via CI
without false positives

(From OE-Core rev: b4afe3c1aaf5e8296e410ef01960f48a09dd717c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit b9551f9180bf9f13fb1c480b5b7892fdc831ffcd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-13 15:23:34 +00:00
Sergei Zhmylev
8a4021d237 wic: make ext2/3/4 images reproducible
Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime.
Currently, all the files are being added into the rootfs image using
mkfs -d functionality which affects all the timestamps excluding mtime.
This patch ensures these timestamps inside the FS image equal to
the SOURCE_DATE_EPOCH if it is set.

(From OE-Core rev: da2c64b3158c58eb0a484d3acbdf0419df2d34e8)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 75d2dd0ea7790db2e8ee921784ca373abff2df65)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-24 15:30:01 +00:00
ciarancourtney
f77e16ea7d wic: swap partitions are not added to fstab
- Regression in 7aa678ce804c21dc1dc51b9be442671bc33c4041

(From OE-Core rev: 397676b3ad152b209916b152f1b77e772a2af14b)

Signed-off-by: Ciaran Courtney <ciaran.courtney@activeenergy.ie>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit f1243572ad6b6303fe562e4eb7a9826fd51ea3c3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20 08:19:17 +00:00
Sergei Zhmylev
7a31cf96eb wic: honor the SOURCE_DATE_EPOCH in case of updated fstab
In case user requested to build a binary repeatable package,
it's required to honor the SOURCE_DATE_EPOCH environment
variable. So forcefully set mtime inside all the routines
which modify fstab in case it is updated.

(From OE-Core rev: 2671667f33d0eac1425db3fc4dff56d4eed1eb3c)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 99719a3712a88dce8450994d995803e126e49115)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-10 14:39:24 +00:00
Joshua Watt
67b2d0267a runqemu: Fix gl-es argument from causing other arguments to be ignored
The code to parse arguments was inadvertently skipping all arguments in
the elif block after gl-es if it was specified on the command line.

(From OE-Core rev: 45356f2ef90e4b67b890ca745513fafa32a469cf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 718bb8d56f6a24c86e67830a7d13af54df2ebb4e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit dd1dcfada1fa46ecb8227c2852769b35026875d3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Joshua Watt
b9f2ebe163 runqemu: Do not perturb script environment
Instead of changing the script environment to affect the child
processes, make a copy of the environment with modifications and pass
that to subprocess.

Specifically, when dri rendering is enabled, LD_PRELOAD was being passed
to all processes created by the script which resulted in other commands
(e.g. stty) exiting with a failure like:

 /bin/sh: symbol lookup error: sysroots-uninative/x86_64-linux/lib/librt.so.1: undefined symbol: __libc_unwind_link_get, version GLIBC_PRIVATE

Making a copy of the environment fixes this because the LD_PRELOAD is
now only passed to qemu itself.

(From OE-Core rev: 74911cf7ea703c54920a6c58c344a22a46398b02)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 2232599d330bd5f2a9e206b490196569ad855de8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Ross Burton
0893b67988 scripts/oe-check-sstate: force build to run for all targets, specifically populate_sysroot
Since the commit "populate_sdk_base/images: Drop use of 'meta' class and
hence do_build dependencies"[1], builds of images or SDKs don't
recursively depend on the top-level do_build target.  This is typically
a good thing: images just depend on the packages themselves and those
dependencies already exist, but they don't need each recipes sysroot to
be populated.

However, eSDK generation is partly done via the script oe-check-sstate,
which does a 'dry-run' build of the target and collates all of the
sstate that is used.  With this commit the sstate that is used is a
fraction of what would be needed in the SDK, specifically there are no
sysroots populated during the build, so there are no sysroots in the
SDK.

This is obviously a problem, as the entire point of an eSDK is to
contain a sysroot.  Resolve this problem by forcing bitbake to run the
build task for all targets, so that all potentially needed sstate is
collated.

[YOCTO #14626]

[1] 41d7f1aa2c

Tested-by: Andrej Valek <andrej.valek@siemens.com>
(From OE-Core rev: c6b8543fbd0e840483cbcdca93116cc9c994a9f2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1b62344f919b5122f048b6409d09386d7d6dd3cd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Ross Burton
ea5e17bf05 scripts/oe-check-sstate: cleanup
The scriptutils import isn't used, there's no need to run bitbake
in a shell environment, and invoke bitbake as a list instead of a
string.

(From OE-Core rev: 4b9946bd3961679048e1460bdfc74b286c386feb)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 663aa284adf312eb5c8a471e5dbff2634e87897d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Martin Jansa
a0b0b25a82 create-pull-request: don't switch the git remote protocol to git://
Many git repos prefer https:// nowadays and many removed support
for git://.

This breaks the script when using github.com even when selected remote
is ssh (git@github.com:openembedded/...), it will re-write it to git://
before calling git pull-request causing:

openembedded-core $ scripts/create-pull-request -u github -b jansa/artifacts -o pull-kernel
NOTE: Assuming local branch HEAD, use -l to override.
fatal: unable to connect to github.com:
github.com[0: 140.82.121.3]: errno=Connection timed out

warn: No match for commit ea003bd026aa24bb4c8b7562f44ed6512e921259 found at git://github.com/shr-distribution/oe-core
warn: Are you sure you pushed 'jansa/artifacts' there?
ERROR: git request-pull reported an error

(From OE-Core rev: 5aac691040502e6f578316dfc090643d341dd3cb)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 64c466920b808c35d1ac87b47cf438bc79becea7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-11 21:56:13 +01:00
Alexandre Belloni
a5eeb959bf runqemu: display host uptime when starting
In order to be able to debug issues on the host that have an impact on the
guest, it is useful to get the uptime of the host while starting so we can
match with the events in dmesg. Also include the uptime when cleaning up.

(From OE-Core rev: 08406e03abddc7290c0c2296aa179725a58155d3)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2d96499823f7de6e16a461426491e015ba63c1ec)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-16 17:53:23 +01:00
Ulrich Ölmann
2a82c3d93a scripts/runqemu.README: fix typos and trailing whitespaces
(From OE-Core rev: b765d234c9d9d1e5e6bb02b074f42cea8fb48df2)

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 217b00d378f359689613ca4c0666bb2eed040f69)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-12 08:41:48 +01:00
Ross Burton
cd88865fd2 wic/bootimg-efi: use cross objcopy when building unified kernel image
We can't rely on the host objcopy knowing how to process target binaries,
so use the cross objcopy in the sysroot instead.

Also construct the command argument-by-argument as the format expression
was getting unwieldy.

(From OE-Core rev: 0264aeedbf21e9e7a104243c11b3b57f00e38bda)

(From OE-Core rev: 4e8b803d7efa13c950353bb00ab65be22eb61736)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-31 16:54:17 +01:00
Ross Burton
8191b79123 wic: add target tools to PATH when executing native commands
We might want to run a cross tool, such as objcopy, in wic.  These are
in a TARGET_SYS/ subdirectory under /usr/bin, so add that directory to
the search path too.

(From OE-Core rev: c523549141e5c31edc75281f581d97867b7d251d)

(From OE-Core rev: f8e0512503410ca5137fcf114fbffb52aa98be07)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-31 16:54:17 +01:00
Paul Eggleton
84eb7f3b98 relocate_sdk.py: ensure interpreter size error causes relocation to fail
If there is insufficent space to change the interpreter, we were
printing an error here but the overall script did not return an error
code, and thus the SDK installation appeared to succeed - but some of
the binaries will not be in a working state. Allow the relocation to
proceed (so we still get a full list of the failures) but error out at
the end so that the installation is halted.

(From OE-Core rev: 3f258378bcaebb9c42c0011c7c424c920ef71d22)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c5a9a448e462d3e5457e8403c5a1a54148ecd224)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
ac8186042d scripts/oe-setup-builddir: make it known where configurations come from
"with some default values" isn't clear; if the user wants to change or
inspect the default values and their history, we should help them find
where they are. This becomes especially important when using template
configs other than poky's.

(From OE-Core rev: b60f4c7f3205b354469c1aa8b56ceaacc11d486b)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit ec34783ffc34eb9e9697f1b192c5a0043f1ca2c6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Mark Hatle
cb87867c47 runqemu: Add missing space on default display option
(From OE-Core rev: fcb55a198eddf4110fd4baf67614a7598441d952)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ee9428611fc38bc711b5b3e12cf0d3257b1b5680)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
0ff2b0bbf8 devtool/upgrade: catch bb.fetch2.decodeurl errors
Otherwise, workspace cleanup (removing bogus recipe and source tree)
will not happen, leaving breakage behind.

(From OE-Core rev: 72bfdca08029c031cedc9dbbf366663632c1c8db)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 74774f9b67580a8c56f605dfd4cc7b856bbeeae8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
53019423e4 devtool/upgrade: correctly clean up when recipe filename isn't yet known
There is a coding error in the second invocation of _upgrade_error:
rf is passed into it before it is initialized in the try: block. And so
bogus recipes are left behind in the workspace, causing breakage.

Instead, rewrite the functions to take the recipe directory name in the
workspace layer, which can be calculated in advance.

(From OE-Core rev: a7d406dce577192f98d76dbae411c72a8b7f07d4)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit e653996369c1d2b5ac8367ad85f4816d679b6c98)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Roland Hieber
58d3e1b897 devtool: error out when workspace is using old override syntax
When the workspace bbappends are still using the old override syntax
with EXTERNALSRC_pn-*, externalsrc_re will not match, and pn will never
be assigned, leading to a nondescript UnboundLocalError being raised on
the user's terminal. Try to detect that situation and give the user a
hint how to solve it.

(From OE-Core rev: 3aee3425956020166d99ec085e35e21b3daf625f)

Signed-off-by: Roland Hieber <rhi@pengutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d42ea8e849cf2df3708406418b961168268b316a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Mihai Lindner
f327cf1077 wic/plugins/rootfs: Fix NameError for 'orig_path'
Fix "NameError: name 'orig_path' is not defined".
It's a typo from when this error was handled outside this function.

(From OE-Core rev: 15015dda2cfccb9d2894f8d9d8f04043604b7a5e)

Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2124ec0d9f9de2da476f0024a0ccf70da987420f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:29:15 +01:00
Paul Eggleton
ee60960da3 devtool: finish: handle patching when S points to subdir of a git repo
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.

(From OE-Core rev: 7cf53810b1dc3d14c4838a610b3d53170f552c19)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad3736d9ca14cac14a7da22c1cfdeda219665e6f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Paul Eggleton
de0d04dfa3 devtool: ignore pn- overrides when determining SRC_URI overrides
If (perhaps foolishly) at your configuration level you have e.g.

  SRC_URI_append_pn-recipename = " file://patchname.patch"

and then run devtool modify on a different recipe, an error occurs:

  INFO: SRC_URI contains some conditional appends/prepends - will create branches to represent these
  ...
  ERROR: [Errno 2] No such file or directory: '/path/to/downloads/patchname.patch'

pn- overrides would not constitute an alternative configuration that we
should handle in this context, so just ignore them to avoid the issue.

(From OE-Core rev: 9beb3472624050593fc30d5a00d3d13fec4441df)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3f2a812ade42ece0bb59b2d303125a91b29936dd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Thomas Roos
0b047cb6e7 recipetool/devtool: Fix python egg whitespace issues in PACKAGECONFIG
Substitute expressions or whitespace from python egg requires.txt when
generating PACKAGECONFIG

Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements.
Recipetool parses this information to generate the PACKAGECONFIG.
These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG.
This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable.
Also adding an oe-selftest for this.

[YOCTO #14446]

(From OE-Core rev: 5a1fd88439c28c473a1723a040d780f100d6295e)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a854d95a79e64f3f82abfa4cc1daec750abf4249)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 08:27:16 +01:00
Martin Jansa
2fd4a20088 wic: fix WicError message
* add missing % to print the values instead of:
  | INFO: Build artifacts not found, exiting.
  | INFO:   (Please check that the build artifacts for the machine
  | INFO:    selected in local.conf actually exist and that they
  | INFO:    are the correct artifacts for the image (.wks file)).
  |
  | ERROR: ("The artifact that couldn't be found was %s:\n  %s", 'kernel-dir', '/OE/build/deploy/images/qemux86-64')

(From OE-Core rev: 772c9f66633e85c5059670d328e1b5fad407457f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e104c2b1273d8c5bd97893f318bf2a2699ef7f2d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 08:27:16 +01:00
Xiaobing Luo
38f2a77176 devtool: Fix _copy_file() TypeError
when devtool finish, the _copy_file() failed.
--------------------------------------------
TypeError: _copy_file() got an unexpected keyword argument
'base_outdir'
--------------------------------------------

Fixes: 05f2d5d2ce00 ("devtool: finish: add dry-run option")

(From OE-Core rev: 5e6f4d0d3d314897b8ab2f45b3a78b0da9df99ab)

Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a45d9dc089fb2719ca69b92870917f8c0925f632)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-22 23:46:28 +01:00
Felix Moessbauer
a8a094e511 wic/plugins/rootfs: Fix permissions when splitting rootfs folders across partitions
This patches makes locating the file database containing the file and
folder usernames and permissions more reliable.
In addition to locating it relative to the partition directory, we also
try to locate it relative to the IMAGE_ROOTFS.

Prior to this patch, the database was not found when using
--rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file
permissions and ownership.

(From OE-Core rev: a5406116ef647ff0cabc6b9466aa0e34e6a20750)

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 09e18ee246da8b56f446c4db548fb9c7e895142b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-20 10:08:06 +01:00
Richard Purdie
7868ccd57a scripts/git: Ensure we don't have circular references
This is horrible but I'm running out of better ideas. We hit circular reference
issues which we were trying to avoid in the core HOSTTOOLS code. When building
the eSDK, there can be two copies of the script.

Therefore assume git will never be in a directory called scripts. This
fixes eSDK build failures.

(From OE-Core rev: b9dcaa76b3274ced1e4b9e2ca33f778e8cd50032)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 27de610ac30d4c81352efc794df7e9b1060f7a68)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Richard Purdie
db10b2623d scripts: Make git intercept global
The previous minimially invasive git intercept simply isn't enough. For example,
meson used in the igt-gpu-tools recipe hardcodes the path to git in the configure
step so at install time, changing PATH has no effect.

There are lots of interesting things we could do to try and avoid problems but
making the git intercept and dropping fakeroot privs for git global is probably
the least worst solution at this point. It will add slight overhead to git calls
but we don't make many so the overall impact is likely minimal.

(From OE-Core rev: 07f282fb94a5a7c0a3fad451c844e5b26074c744)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit af27c81eaf68ee681dcd9456a74cca6a9ab40bf6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Steve Sakoman
4280a7dd25 scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for phantomjs and optipng
Use of those tools was removed in b5c131006e3fad0a15e6cdf81f71dc1e96647028
perf-build-test/report: Drop phantomjs and html email reports support

(From OE-Core rev: 1063525be9b040ece8636c03ac8bab13952eb561)

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 33df447affa7a3a360b1da028e6b12fbcd388db6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:05 +01:00
Paul Gortmaker
f6eb720890 install/devshell: Introduce git intercept script due to fakeroot issues
In a devshell, recent versions of git will complain if the repo is owned
by someone other than the current UID - consider this example:

 ------
  bitbake -c devshell linux-yocto

  [...]

  kernel-source#git branch
  fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else)
  To add an exception for this directory, call:

        git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source
  kernel-source#
 ------

Of course the devshell has UID zero and the "real" UID is for "paul" in
this case.  And so recent git versions complain.

As the whole purpose of the devshell is to invoke a shell where development
can take place, having a non-functional git is clearly unacceptable.

Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I
suggested we probably will see other similar instances like this and should
make use of PATH to intercept via devshell wrappers - conveniently we already
have examples of this.

Here, we copy the existing "ar" example and tune it to the needs of git to
combine Richard's suggestion and mine.

As such we now also can store commit logs and use send-email with our user
specific settings, instead of "root", so in additon to fixing basic
commands like "git branch" it should also increase general usefulness.

RP: Tweaked the patch so the PATH change only applies to the devshell task
and is a generic git intercept rather than devshell specific.

RP: Also apply the PATH change to do_install tasks since that also runs under
fakeroot and several software projects inject "git describe" output into
their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing
reproducibility issues from systems with different git versions.

(From OE-Core rev: 3a320c1555bf39b2d3c218ffc36827d9dda60fe1)

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3266c327dfa186791e0f1e2ad63c6f5d39714814)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-04 13:07:34 +01:00