Commit Graph

151 Commits

Author SHA1 Message Date
Joshua Watt
5254f0af57 classes/populate_sdk_base: Add setscene tasks
do_populate_sdk was added to SSTATETASKS, but had no _setscene task
created to allow it to actually run from sstate. Add it so that SDKs can
be restored from sstate.

Note that like do_image_complete, do_populate_sdk is marked with
SSTATE_SKIP_CREATION by default so sstate is not used for them; adding
this task will allow it to work if the user overrides this default
though.

(From OE-Core rev: 292cd79bfb9a9e62f1cb4afaef7d8c7f2c4aac98)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1f204592903a2fd9375b0f3c9c52e7dde0467460)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-08 23:41:00 +00:00
Richard Purdie
ad7cdc4f01 populate_sdk_base/images: Drop use of 'meta' class and hence do_build dependencies
The 'meta' is old and not very useful. It empties PACKAGES and creates
recursive do_build dependencies.

We shouldn't need such recursive build dependencies any more so simplify the code.

This does cause behaviour changes as some dependencies are no longer built.
It did show up issues with qemu-helper-native handling for example but those
issues look like real races and the underlying dependency chains were not correct.

'bitbake XXX --runall build' would give the previous behaviour.

(From OE-Core rev: 41d7f1aa2cc9ef5dba4db38435402d4c9c0a63e1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23 13:49:11 +01:00
Richard Purdie
f62f684de8 sdk: Decouple default install path from built in path
Add SDKPATHINSTALL which is used as the default install location of the SDK
instead of SDKPATH. This means the default install path isn't encoded into
every SDK binary, meaning if a date is used there the entire SDK doesn't
have to rebuild. Most distros can switch to only customise SDKPATHINSTALL
meaning more sstate reuse too.

[YOCTO #14100]

(From OE-Core rev: bc4ee5453560dcefc4a4ecc5657df5cc1666e153)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:11 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Khem Raj
bb30baceec sdk: Enable do_populate_sdk with multilibs
This patch enables building image based SDKs for multi-libbed images
e.g. lib32-core-image-minimal and so on. Change the path to nativesdk
tools to use recipe-sysroot since thats where the nativesdk components
are installed and it will need access to qemu wrappers during build for
processing intercepts

[YOCTO #14444]

(From OE-Core rev: 6196a785eababb040ee1dee9f33cb6d6dad77eef)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-22 15:18:11 +01:00
Christopher Larson
13e372dfd9 image,populate_sdk_base: move 'func' flag setting for sdk command vars
Setting the 'func' flag on the commands variables ensures that they are parsed
as shell, and therefore that the referenced commands contents are included in
checksums. Doing this only in image.bbclass means that this is missing in
recipes that are not images, but which inherit populate_sdk or populate_sdk_base
directly, so move it to the latter.

[YOCTO #13998]

(From OE-Core rev: edc28907ce19a7298059dd388933c58a9c6c28b9)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-31 08:51:02 +01:00
Richard Purdie
9ba9f879e1 populate_sdk: Add directories to PSEUDO_IGNORE_PATHS
Some paths used in sdk construction shouldn't be tracked under pseudo
so list these.

(From OE-Core rev: 21621e09162ff9e8014a1cfba78d0f2c3746202a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27 22:40:41 +00:00
Tomasz Dziendzielski
1d3e7c7832 populate_sdk_base: Fix condition syntax if SDK_RELOCATE_AFTER_INSTALL is disabled
If variable is set to empty string the comparison is "if [ -eq 1 ]"
which fails with "[: -eq: unary operator expected".

(From OE-Core rev: 36a2dc83fd0e1d6e2b8441e24a4cbc48a6c4fc19)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-24 08:25:14 +00:00
zangrc
400359f1d2 classes: Fixed the problem of undefined variables when compiling meta-toolchain.
The product name when compiling meta-toolchain is:
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.host.manifest
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.target.manifest
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.testdata.json
poky-glibc-x86_64--cortexa57-qemuarm64-toolchain-3.1+snapshot.sh

inherit image-artifact-names to solve this problem.

(From OE-Core rev: 1a566f1685258bdd6da50c8d2183ff21bed41bb3)

Signed-off-by: Zang Ruochen <zangrc.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 12:34:29 +01:00
Richard Purdie
a2db4fa127 base/bitbake.conf: Enable pseudo path filtering
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,

This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.

There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.

This patch fixes a few corner cases in OE-Core when used with this new
ignore list:

* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
  to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
  user so we don't want the file ownership information in the hashequiv outhash
  calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
  so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
  by bitbake. We therefore need to trigger reparsing if this changes, which means
  SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
  Rework the variables to handle this. This otherwise breaks some of our sstate
  tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.

(From OE-Core rev: ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07 11:18:38 +01:00
wenlin.kang@windriver.com
f63e51a165 populate_sdk_base.bbclass: fix warning: name not matched
When do populate_sdk with SDK_ARCHIVE_TYPE = "zip", have below warning
in log file, this patch fixes it.

zip warning: name not matched: sysroots/core2-64-poky-linux/etc/tmpfiles.d/etc.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/tmpfiles.d/home.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/resolv-conf.systemd
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/mtab
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/resolv.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/var/lock
zip warning: name not matched: sysroots/core2-64-poky-linux/usr/lib/environment.d/99-environment.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/sbin/klogd
zip warning: name not matched: sysroots/core2-64-poky-linux/sbin/syslogd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chfn
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chsh
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/passwd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chpasswd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/vipw
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/vigr
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/nologin

(From OE-Core rev: 5f7df08aca81ac269490876fe84c7fc5b1d5739c)

Signed-off-by: Wenlin Kang <wenlin.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-14 14:15:11 +01:00
Ovidiu Panait
2aa51e2e31 populate_sdk_base: create_sdk_files: Use _nativesdk variants for libdir and base_libdir
Use the _nativesdk variants for libdir and base_libdir to fix the resulting
ld.so.conf paths in the SDK for systems with libdir=/lib64.

(From OE-Core rev: a6077e87f4eeb89b0617883c9c01e08219ae44dc)

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 15:02:19 +01:00
Ovidiu Panait
a3b21540c3 populate_sdk_base: Add ld.so.conf for nativesdk-binutils
Extend the functionality provided by commit [1] to the SDK as well. This way we
can make sure that nativesdk-binutils finds SDK libraries first rather than
host ones.

This is useful for example when trying to build the linux kernel using
nativesdk-gcc. This scenario currently fails because it tries to link to host
libraries rather than SDK host ones:

make x86_64_defconfig
make bzImage
...
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile:1101: recipe for target 'prepare-objtool' failed
make: *** [prepare-objtool] Error 1
....
/../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
...

[1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]

(From OE-Core rev: 7ec84a463ad4c45aee9cd2cbc75b43e5aab5cd18)

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 15:02:19 +01:00
Mike Looijmans
27ff81bfd8 classes/populate_sdk_base: Implement xz compression options
Building an SDK on a machine with 8GB RAM resulted in excessive swapping
due to the xz compressor using ~20GB of memory. This is because xz is
being called with "-T 0 -9".

To allow tuning the compression versus memory usage, introduce a variable
named SDK_XZ_OPTIONS that defaults to a more sane default:
SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${SDK_XZ_COMPRESSION_LEVEL}"
The use of XZ_DEFAULTS fixes the excessive memory usage.
The SDK_XZ_COMPRESSION_LEVEL variable allows overriding the speed vs
compression. In an office or development environment the extra time
spent on compressing a few percent more is just not worth it.

(From OE-Core rev: 25533a48a19e3e6eb9f96be51b1373830b5bb9f0)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-22 10:48:43 +00:00
Mark Hatle
35d68aea95 populate_sdk_base: if the SDKIMAGE_FEATURES changes, refresh the SDK
Since the features are processed by a python fragment, we need to explicitly
list the variables that should affect the resulting hash, and thus sstate
re-use.

(From OE-Core rev: c7c12efba87c803d94a7c9c2f8c98b66aeba6d43)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Joe Slater
4457c74ee5 bash-completion: add image feature
Create bash-completion-pkgs image feature to load *-bash-completion packages
into an image.  The packages are created by the bash-completion
bbclass but are currently never loaded.

(From OE-Core rev: d47c5981877555a5a6b9aeb93ea6f3c1aa97fc44)

Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02 20:22:25 +01:00
Khem Raj
610056167b populate_sdk_base.bbclass: nativesdk-glibc-locale is required on musl too
musl based extensible sdk fails to install

ERROR:  OE-core's config sanity checker detected a potential misconfiguration.
    Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
    Following is the list of potential problems / advisories:

    Your system needs to support the en_US.UTF-8 locale.

The real reason is that its missing locale-archive in buildtools-tarball
hence python3 from buildtools-tarball fails to run bitbake when
installing esdk

(From OE-Core rev: c6e8e13f910544b2a4435d8b9e6dbc65847ef182)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16 23:02:45 +01:00
Richard Purdie
5ef5ece35b populate_sdk_base: Use highest compression level for xz
Whilst this may take longer to compress, it reduces network transfer and potentially
extraction time so is worthwhile.

(From OE-Core rev: 0a89d486cf596da791c27cff14ef3d6ca1cbb75a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-29 12:54:12 +01:00
Changqing Li
3418d1326c populate_sdk_base: provide options to set sdk type
Current sdk type is tar.xz, but for mingw sdk, since we
have symlink under the sdk folder, 7zip which used to
extract tar.xz cannot handle it, refer 7zip upstream bug:
https://sourceforge.net/p/sevenzip/discussion/45797/thread/c71d6b96/
so add option for usr can select the sdk type.

Add override SDK_ARCHIVE_TYPE, default type is tar.xz, and also
support type zip. user want to use zip type can set SDK_ARCHIVE_TYPE
to zip.

(From OE-Core rev: 57a33048a89a422cfdc986d3489c67b2d297e1e7)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-03 06:11:57 +01:00
Adrian Bunk
0734405c9a Use the best xz compression for the SDK
It saves 23% space for me, and decompression time is also shorter.

Compression time and xz memory usage should be less of a worry
for the SDK.

(From OE-Core rev: 353d93ead899a479fc6bc3625edc87269a891d39)

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26 10:09:08 +01:00
Khem Raj
c6a243dbb2 image_types.bbclass: Set memory usage limit and CPU threads for xz
when building with opkg backend and huge packages e.g. chromium/llvm all
going in parallel, memory pressure causes xz to catapult with

do_package_write_ipk: Failed to create package, opkg-build failed with: xz: (stdin): Cannot allocate memory

since there are many tasks going on in parallel, xz adds to memory pressure
and it wants it all, put an upper limit for memory xz can use

We add a variable XZ_MAXRAM with 30% of RAM limit and can be customized
if builders have more memory one can set it like

XZ_DEFAULTS = "-M 0 -T 0"

(From OE-Core rev: 28b277a93a34bba033d9d0d9f3227c9453efd384)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-14 11:20:55 +00:00
Joshua Watt
ba3aa53112 Change default debug split to make separate source packages
Changes the default PACKAGE_DEBUG_SPLIT_STYLE to generate separate
source and debug packages. SDKIMAGE_FEATURES is updated to include the
source packages so that there is not change for the SDK contents.

[YOCTO #12931]

(From OE-Core rev: 8df14b678e44cc749b361224af05ccbcfa9ae9b5)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:09 +00:00
Joshua Watt
24ff4ee9fe classes/populate_sdk_base: Add src-pkgs to COMPLEMENTARY_GLOB
Add a COMPLEMENTARY_GLOB for source packages (*-src) so that they can be
included when PACKAGE_DEBUG_SPLIT_STYLE is "debug-with-srcpkg"

[YOCTO #12931]

(From OE-Core rev: d6d2175032c6c950a0d8d63dad0734e1a9f80e12)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-22 14:35:58 +00:00
Richard Purdie
e1b639bd5e meta: Fix python code quoting issues
python 3.8 will be stricter about python quoting. Fix up several misquoted
expressions and fix Deprecation warnings like:

Var <do_compile>:1: DeprecationWarning: invalid escape sequence \$

(From OE-Core rev: 3ba6cee84de89f8eb200e4c93d446f6cdeeaa4be)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
Andrej Valek
5f30534c47 rootfs: possibility to add packages to debug rootfs
Static libraries are not included in rootfs, it means, that sources are
not going into debug rootfs. This option enables to install additional
packages even if the standard package is not installed.

(From OE-Core rev: 708fe1df3d3e22dd693ae7bbfdd3e5af1a1b0bdc)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18 10:56:55 +01:00
Andrej Valek
38a4f6e4f8 populate_sdk_base: move template files location to variables
Let users to use custom toolchain template files instead of overloading
the whole function.

(From OE-Core rev: d4d5fdadaf2f671559326ab594fb596995846ca4)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:35 +01:00
Kai Kang
1d5b01c956 target-sdk-provides-dummy: skip package_qa_multilib check
The rprovides of target-sdk-provides-dummy don't be updated with
multilib, so it fails package_qa_multilib check. Because
target-sdk-provides-dummy doesn't install any file to sysroot, it is
safe to skip package_qa_multilib check for target-sdk-provides-dummy.

Remove ${MLPREFIX}target-sdk-provides-dummy from TOOLCHAIN_TARGET_TASK
at same time in populate_sdk_base.bbclass.

(From OE-Core rev: 3197c086269a4b21fb807a9c552b56f23c5b86dc)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Richard Purdie
7a76bb5304 populate_base_sdk: Stop running gcc --version all the time
Running 'gcc --version' for every image recipe is slow and increases parsing
time/resource usage for no good reason. Only compute the value in when we're
really running the task/function.

(From OE-Core rev: bf49316bb9913b7c89de64d6a194be31aa66e16b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 18:00:25 +01:00
Christopher Larson
762e6b7b45 populate_sdk_base.bbclass: inherit and use image-postinst-intercepts
(From OE-Core rev: fb83997ded3789c7447402a9fda03b1669cecae0)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16 09:49:39 +01:00
Richard Purdie
d8762a1ea7 Revert "relocate_sdk.py: remove hardcoded SDK path"
This reverts commit 6671a4d980c8bef8f402780a308f6c43a25044aa.

This breaks uninative tarball since the call of relocate_sdk.py from uninative.bbclass
wasn't updated to account for this change. It isn't clear what value that code could
pass in and this isn't simple to fix so revert until a better fix can be found
that doesn't break uninative.

(From OE-Core rev: b247392b4ced57cfe694656032f6a6723740a9e8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:46:56 +01:00
Ruslan Bilovol
bf5b43495a relocate_sdk.py: remove hardcoded SDK path
This patch removes hardcodes added to relocate_sdk.py
during SDK build, making it flexible and reusable.
Now default SDK path is passed to the script as
parameter rather then harcoded inside it.

This allows to reuse this script for multiple
relocations, and adds possibility to relocate
SDK multiple times

(From OE-Core rev: 6671a4d980c8bef8f402780a308f6c43a25044aa)

Signed-off-by: Ruslan Bilovol <rbilovol@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Richard Purdie
dd31a246aa populate_sdk_base: Use default value assignment for SDK_TITLE/SDKEXTPATH
This allows the user mode flexibility about centrally overriding these
rather than needing to do it on a per recipe basis.

(From OE-Core rev: b0fb5bbd5f62857c32b2e071bcac841f856b1f1b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Alexander Kanavin
bdfca89b80 image.bbclass, populate_sdk_base.bbclass: make all variants of qemuwrapper-cross available in sysroots
The variants are needed in particular when executing postinst_intercepts as
those may require running binaries built for different architectures and
against different sets of library paths, when multilib is in use (or nativesdk host
packages are installed), so a single global variant of the script was not working.

I do understand expanding PATH and DEPENDS in this manner is hackish, however
every other approach I could think of is worse.

(From OE-Core rev: 2f31eecc40ea4d0865aa28d65a0ba7d5a629393a)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:58 +01:00
Ross Burton
8b50dfcecf populate_sdk_base: use xz -T instead of pixz
xz has native support for threaded compression now and SDK creation was the only
part of oe-core which is using pixz instead of xz.

Not only does this remove pixz-native from the SDK dependencies, but in my
limited testing xz -T0 is slightly faster and produces smaller archives than
pixz for the same input.

(From OE-Core rev: ce1cfa57d9828c0898f52e736f671ce8db534031)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:06 +01:00
Peter Kjellerstedt
144faeb10c classes: Use ${MACHINE_ARCH} instead of ${MACHINE} for stamp-extra-info task flag
Without this change, there will be two sstate index files in
tmp/sstate-control for any machine that contains a dash in the
name.

(From OE-Core rev: 29e7799bdb3773c40492e01448e0c614ed44583d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Alexander Kanavin
e4b8c2bd20 sdk.py: run postinst intercepts
Previously this wasn't done, and so any packages installed from
populate_sdk would not have the postinsts fully executed
(particularly generation of various caches via running nativesdk or target
binaries with qemu wasn't working).

[YOCTO #12630]

(From OE-Core rev: a484ff072eea3f47de2c3348048201249cefa46b)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Ross Burton
dc7ec9b8e0 populate_sdk_base: depend on nativesdk-glibc-locale
If we're building a SDK and we're using glibc so may be installing locales,
add a build-dependency on natiesdk-glibc-locale so the locales we need will
exist.

(From OE-Core rev: 8d6869a0a89d8cf3c6e57723fab2750ba2c885db)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-06 06:19:18 -08:00
Ross Burton
2f56415059 populate_sdk: install UTF-8 locales in SDKs
As glibc 2.27 can't read older locale-archives, SDKs using glibc 2.27 on hosts
using glibc earlier than 2.27 won't be able to find any locales, so bitbake
won't start and Python can't use UTF-8.

So by default install all locales into the SDK.  Special-case Extensible SDKs by
installing no locales as they ship glibc in a buildtools, and that will have the
locales.

Locale installation requires cross-localedef, so add that to DEPENDS.

Also remove the explicit en_US addition in buildtools-tarball as it is now
redundant.

(From OE-Core rev: 96896568d197cd06302713c24c0f7d91bfaea6c1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:18:47 +00:00
Richard Purdie
0d0984e1e6 nativesdk/sdk: Update sdk dummy providers
When we migrated rpm v5 -> v4, we lost the ability to drop "per file"
dependencies from the rpm backend for things like "/bin/bash" and
"/usr/bin/env" which meant the sdks were becomming 'bloated'.

This restores the functionality using a dummy package, similarly to
the way the buildtools perl issue was addressed. It also removes
the non-functional old code so as not to confuse people in future.

I ran into this problem trying to filter dependencies to only rpms
a build directly depends upon and it turns out we have some determinism issues
in this area so this is something key to fix.

(From OE-Core rev: 9d490dc01dcedb216129b22cbe17a6c99efc4f5c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-11 10:26:06 +00:00
Yi Zhao
3c901c5933 populate_sdk_base.bbclass: add handling package exclusions to do_populate_sdk
There is an error for populate_sdk when we exclude package by using
PACKAGE_EXCLUDE.

Reproduced steps:
echo "PACKAGE_EXCLUDE = \"psplash\"" >> conf/local.conf
bitbake core-image-sato -c populate_sdk

Error log:
ERROR: core-image-sato-1.0-r0 do_populate_sdk: Could not invoke dnf.
Command '/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/recipe-sysroot-native/usr/bin/dnf -y -c
/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/sdk/image/opt/poky/2.3/sysroots/i586-poky-linux/etc/dnf/dnf.conf
--setopt=reposdir=/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/sdk/image/opt/poky/2.3/sysroots/i586-poky-linux/etc/yum.repos.d
--repofrompath=oe-repo,/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/oe-rootfs-repo
--installroot=/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/sdk/image/opt/poky/2.3/sysroots/i586-poky-linux
--setopt=logdir=/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp
-x psplash --nogpgcheck install run-postinsts dnf
packagegroup-core-x11-base packagegroup-core-ssh-dropbear
packagegroup-core-standalone-sdk-target packagegroup-base-extended
packagegroup-core-x11-sato psplash packagegroup-core-boot rpm' returned 1:
Unable to detect release version (use '--releasever' to specify release version)
Added oe-repo repo from
/buildarea1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/oe-rootfs-repo
Last metadata expiration check: 0:00:00 ago on Thu 10 Aug 2017 09:26:32 AM UTC.
No package psplash available.
Error: Unable to find a match

Add handling package exclusions to do_populate_sdk. The code copies from
do_rootfs in image.bbclass.

(From OE-Core rev: 86db855da4ee000737281ef7cc893d56854b3952)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:13 +01:00
Aníbal Limón
2d84c7a55d classes/populate_sdk_base: Fix SDK manifest generation
The write_{host,target}_sdk_manifest don't need to be set as
SDK_POSTPROCESS_COMMAND because are already append to
POPULATE_SDK_POST_{HOST,TARGET}_COMMAND and the if doesn't
make sense because are only added for populate sdk task.

[YOCTO #11741]

(From OE-Core rev: b10ecbab3acd46e48d36910e30544e9f5f08d6d7)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-06 14:38:14 +01:00
Amarnath Valluri
a4ada62a21 meta/lib/oe/sdk.py: support added for executing pre-target commands
Added a new POPULATE_SDK_PRE_TARGET_COMMAND variable, which can contain
functions need to be executed at pre traget sysroot creation phase.

classes/populate_sdk_base.bbclass: Added POPULATE_SDK_PRE_TARGET_COMMAND to sdk
command variables list.

(From OE-Core rev: b0c65c8a64cd0b77629c9f3c65fc827d4cdcf026)

Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-06 14:38:13 +01:00
Francisco Pedraza
1b804d7944 meta/classes/populate_sdk: Adds support for generating eSDK manifest files
Add get_extra_sdk_info to reuse code in buildhistory

The functionalities to generate SDK and eSDK manifest files are different,
the SDK comes from package information and the eSDK comes from sstate artifacts.
Only execute write_sdk_{host, target}_manifest when is on populate_sdk class.

Adds new functions write_sdk{host, target}_ext_manifest to execute on postprocess
in populate_sdk_ext because at the end we have all the sstate artifacts to
generate the manifest.

[YOCTO #9038]

(From OE-Core rev: 25ad7ed6f7bb0c931b404bda09576323200d093d)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12 23:01:23 +01:00
Robert P. J. Day
f5187871ce classes: Replace "if test" file tests with POSIX file tests
In entire meta/classes/ directory, replace shell tests of the form
"if test -? ..." with POSIX tests of the form "if [ -? ...

(From OE-Core rev: 78928016f4cf38cf6751cb089200bf950d07ae93)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 11:08:34 +01:00
Ming Liu
e2280b9e12 meta: remove remaining True option to getVar calls
This is a complementary fix to commit 7c552996:
[ meta: remove True option to getVar calls ]

it intended to remove all True option to getVar calls, but there are
still some remaining.

Search made with the following regex: getVar ?\((.*), True\)

(From OE-Core rev: 87d03ffe03d6f01e360bfd51714be96e62506e0a)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:02 +00:00
Ed Bartosh
5302fb4620 toolchain-shar-extract: compare SDK and host gcc versions
If ext sdk is built by gcc version higher than host gcc version
and host gcc version is 4.8 or 4.9 the installation is known to
fail due to the way uninative sstate package is built.

It's a known issue and we don't have a way to fix it for above
mentioned combinations of build and host gcc versions.

Detected non-installable combinations of gcc versions and
print an installation error.

[YOCTO #10881]

(From OE-Core rev: bfb9ea2604557804bd8c16adb57ccdd868a4020f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Richard Purdie
7fb3c6a40a image/packagegroup/populate_sdk: Drop do_populate_sysroot task properly
Setting do_populate_sysroot as noexec means the code keeps thinking it can find
a manifest file for it. It also complicates sstate installtion since the code
would believe there is an sstate object there it should look for.

Instead, delete the task. This causes sdk failures as the dependencies are wrong
so fix those as well.

(From OE-Core rev: bd7d0314038a4c1a8e8c9ebdb7194f8e17db3fef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26 10:44:27 +00:00
Aníbal Limón
ad2168365f classes/populate_sdk_base: Add write_sdk_test_data to postprocess
This function will generates testdata.json per SDK type.

[YOCTO #10231]

(From OE-Core rev: bff1b66c6b88fd2d99d14cecba27ee2f7462711f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Ross Burton
1349320ee0 populate_sdk: don't force target debug packages into SDK
TOOLCHAIN_TARGET_TASK doesn't need to explicitly list sdk-target-dbg because if
SDKIMAGE_FEATURES contains dbg-pkgs (as it does by default) then they'll all be
installed anyway.  This means that if the user removes dbg-pkgs from
SDKIMAGE_FEATURES then the SDK correctly doesn't have debug packages in.

[ YOCTO #9078 ]

(From OE-Core rev: 4c711830ae26008f73bbf557964bdb86b8c922da)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05 13:54:05 +00:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00