Commit Graph

93 Commits

Author SHA1 Message Date
Richard Purdie
890e0ab069 python3: Fix ptest output parsing
A previous fix to python added -W and removed -v. This reverts that part of the change
since we're no longer getting ptest results at all. This change back to more
verbose output means we start getting test results again.

(From OE-Core rev: d70c77e4d5ec1d7cb4f134377df11e6475f31062)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11 21:12:48 +01:00
Richard Purdie
fb852a81fc python3: Avoid hanging tests
There are python tests which hang with recent kernels, 5.0 onwards. This causes
ptest to timeout for python3. Disable the problematic test until we better understand
the real cause and fix of the issue (discussions are happening with upstream).

See the patch for details/links.

(From OE-Core rev: 2a48df02a2871635f8235645bfd7f7a3ff0aef31)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11 21:12:48 +01:00
Richard Purdie
57933bd958 openssh/util-linux/python*: Ensure ptest output is unbuffered
We need to run sed with the -u option to ensure the output is unbuffered else
ptest-runner may timeout thinkig things were idle. Busybox doesn't have the -u
option so we need to RDEPEND on sed (which is a good thing to do if we use it
anyway).

Alex Kanavin should get credit for discovering the problem.

(From OE-Core rev: d3ffbebf43c23faa43af81c9ecf6fcaef36d675b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09 13:44:39 +01:00
Ross Burton
9dec37ceef python: improve ptest
Add missing runtime dependencies: the test suite needs proper chmod from
coreutils, and the Europe/Minsk timezone.

Also change run-ptest to use -W (run verbosely on failure) instead of -v (always
verbose) to hopefully reduce the noise in the ptest reports.

(From OE-Core rev: 7574f6850797fb7556d2f8077860b7c1d26ae8ec)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-02 15:24:11 +01:00
Alexander Kanavin
c92e780c85 python3: tweaks to achieve 100% pass rate in ptest
(From OE-Core rev: 22f4f1ef17219448416b7cf2a570c7acd7149a64)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08 10:57:19 +00:00
Alexander Kanavin
e2c3247c23 python3: upgrade to 3.7.2
I took the same approach as the recent perl upgrade: write recipe from scratch,
taking the pieces from the old recipe only when they were proven to be necessary.

The pgo, manifest and ptest features are all preserved.

New features:

- native and target recipes are now unified into one recipe

- check_build_completeness.py runs right after do_compile() and verifies that
all optional modules have been built (a notorious source of regressions)

- a new approach to sysconfig.py and distutils/sysconfig.py returning values
appropriate for native or target builds: we copy the configuration file to a
separate folder, add that folder to sys.path (through environment variable
that differs between native and target builds), and point python to the file
through another environment variable.

There were a few other patches where it was difficult to decide if the patch
is still relevant, and how to test that it works correctly; please add those
as-needed by testing the new python.

(From OE-Core rev: 02714c105426b0d687620913c1a7401b386428b6)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08 10:57:19 +00:00
Alexander Kanavin
2daede4aa8 python3: use ncursesw, not ncurses
So that wide-char functions are accssible.

[YOCTO #13142]

(From OE-Core rev: 042187a65d0622fbd721346b4e55146616c87fd8)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-21 23:44:22 +00:00
Changqing Li
82315924b7 python/python3: use cc_basename to replace CC for checking compiler
When working path contains "clang"/"gcc"/"icc", it might be part of $CC
because of the "--sysroot" parameter. That could cause judgement error
about clang/gcc/icc compilers.

eg: if build under /yocto/builds/xicc/,  bitbake python,  $CC will contains
xicc, will make $CC match *icc, but actuall xicc just folder name.

When "*icc" is matched, below errors are reported when
compiling python/python3:
x86_64-wrs-linux-gcc: error: strict: No such file or directory
x86_64-wrs-linux-gcc: error: unrecognized command line option '-fp-model'

Here use cc_basename to replace CC for checking compiler to avoid such
kind of issue.

(From OE-Core rev: 96383efa2726ed1bf7893332d726112a8552fc24)

Signed-off-by: Li Zhou <li.zhou@windriver.com>

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07 14:38:39 +00:00
Serhey Popovych
9d2ea1ea50 python3: Fix do_configure check platform triplet error (2)
When building for powerpc 32bit with musl following error triggered
from do_configure:

checking for the platform triplet based on compiler characteristics... powerpc-linux-gnu
configure: error: internal configure error for the platform triplet, please file a bug report

This is caused by PLATFORM_TRIPLET != MULTIARCH mismatch since MULTIARCH
in case of musl is powerpc-linux-musl. Since triplet is used as part
module name as described in PEP-3149 to make fix less intrusive alias
powerpc-linux-musl to powerpc-linux-gnu to avoid possible runtime
(e.g. tests) incompatibilities later.

Fix was inspired by commit cda0ef61d373 ("python3: fix do_configure
check platform triplet error").

(From OE-Core rev: b2ec91e12088afa0560aecede587d0970fc64110)

Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15 17:10:52 +00:00
Alejandro Enedino Hernandez Samaniego
55553c2477 create_manifest3: Dont match filenames which contain the directory name for new manifest
When creating a new python3 manifest, there is a corner case on which
the filepath for a certain dependency that was found, could contain
the path of an existing folder, e.g. ${libdir}/python3/xmlrpclib.py
module path contains ${libdir}/python3/xml, this causes an issue where
the dependency doesnt get eventually added on FILES for that module.

This patch checks if the dependency that was found is a directory, if it
is, it checks if it matches one of the existing directories on the
manifest, if it is not, then it checks if the dependency's path (without
the filename) matches one of the directories.

(From OE-Core rev: 59db12fdf294cfab5c1730337d092a75867658f7)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-09 11:08:11 +00:00
Ross Burton
ba89e87d8e python3: drop redundant patch
This patch altered the clean target's behaviour to skip the ipkg-install
directory.  However this directory isn't created by opkg, opkg-utils, or the
package_ipk class; and we don't invoke the clean target as we perform
out-of-tree builds.

(From OE-Core rev: 9f8bd475701e5d797d3ffc1ba97647101ba0b9b0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 12:37:02 +00:00
Alejandro Enedino Hernandez Samaniego
6c524c0906 python3: Adds instructions to the manifest file
While there is a bit of documentation regarding building a new
manifest file for python, it seems that users usually only read
the manifest file.

The manifest file is in JSON format which doesn't allow comments,
hence why instructions were initially put elsewhere.

This patch hacks the call to open the JSON manifest file by using a
marker to trick it into reading only part of the file as the manifest
itself, and keep the other part as comments, which contain instructions
for the user to run the create_manifest task after an upgrade or when
adding a new package.

(From OE-Core rev: 3eab24c6dc095fd2305b9be8467aab1191141e35)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-01 11:38:37 +00:00
Mingli Yu
75d2d6776b python3: add tk support
Add support to enable tk via PACKAGECONFIG.
before this patch:
 # python3
 Python 3.5.6 (default, Nov  8 2018, 04:53:45)
 [GCC 8.2.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import tkinter
 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.5/tkinter/__init__.py", line 35, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
 ImportError: No module named '_tkinter'
 >>>

After this patch, if enable tk in PACKAGECONFIG, then
 # python3
 Python 3.5.6 (default, Nov  8 2018, 03:15:52)
 [GCC 8.2.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import tkinter
 >>>

(From OE-Core rev: 4b781d545e7e0b084201cd6a8fad953b0f231513)

Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Ross Burton
fc3af664f3 python3: add python3-venv to the python3-modules RDEPENDS
(From OE-Core rev: ed5e7541677f6a046f85389cd0c879be3db422cd)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Hugues Kamba
0140a1c2f7 python3: Fix python3-pyvenv run-time dependency
Pyvenv is just a small script that uses venv to create virtual
environments.
https://www.python.org/dev/peps/pep-0405/#creating-virtual-environments

This patch adds the python3-venv module as a self-contained package which
python3-pyvenv must depend on at run-time.

The patch also provides the package python3-pyvenv from the pyhton3-venv
package.This is good for future-proofing since python3-pyvenv has been
deprecated and only python3-venv is now available in Python 3.6.
https://docs.python.org/3/library/venv.html.

Without this patch python3-pyvenv is broken because it is missing the
venv module at run-time. This patch specifies the newly created
python3-venv as a run-time dependency of python3-pyvenv.

(From OE-Core rev: effa141bfce55aab25142ee578c95383c755ad73)

Signed-off-by: Hugues Kamba <hugues.kamba@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Ross Burton
be2592f5c7 python3: add ptest runner
Copy the Python 2 run-ptest script to execute the Python 3 test suite.

(From OE-Core rev: d371ff04628bdf7bac66341aeb9a66d145f66416)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-14 09:20:56 +01:00
Ross Burton
0f117e4aac python3: fix patch headers
(From OE-Core rev: 46bc904bf9cf3b2a01de0ba140b31066f90bfa49)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-08 16:04:12 +01:00
Ross Burton
f51d0a19af python3: move sqlite files into python-sqlite
sqlite3/__init__.py was accidentally included in python3-misc.

(From OE-Core rev: 4e397e16996662bce51608cf5e20448b9803c476)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:49 +01:00
Ross Burton
d21d958c5b python3: remove specal handling of sqite3-tests
This package doesn't exist anymore so the manifest tool doesn't need to handle
it specially.

(From OE-Core rev: 30eadb119a64d8561b946d5b8ee30244caeaf134)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Ross Burton
e6aea815de python3: consolidate tests
Currently the bulk of the tests in python3-tests, some more in
python3-sqlite3-tests, and others in their parent module (such as
python3-ctypes).  This is pointless space usage if we're not planning on running
the tests, so consolidate all the tests into python3-tests.

(From OE-Core rev: 554d3690dab49fee3c93b6b677eb3ef58cff64c1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Ross Burton
2ac291af1f python3: don't sort the manifest in create_manifest
Instead of sorting the entire manifest when it is updated, use OrderedDict to
preserve the order of fields.  This means that packages can be ordered in the
manifest to allow non-trivial FILES assignments (such as a package that picks up
pieces of other packages)

The manifest has been regenerated with the new stable ordering, and
distutils-staticdev moved above distutils so the packaging rules work as
expected.

(From OE-Core rev: 1c67c2146e3644a26367a32885d27a4378f17ac6)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Ross Burton
5679b31bcf python3: fix openssl 1.1.1 changes
Due to human error an older revision of the SSL patch was merged.

(From OE-Core rev: 325af0f4a821971a7aeeca35b10e3558f86029e0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 10:04:40 +01:00
Anuj Mittal
3c32b1525a python3{,-native}: backport openssl 1.1.1 compatibility changes
Backport changes from 3.7/3.6 to fix failing python3 ssl test suite.

Fixes [YOCTO #12919]

(From OE-Core rev: 6c123468b546931de005cf136d98bca6b893b37b)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Ross Burton
cb4134f798 python3: fix multiprocessing
glibc 2.28 slightly changed the behaviour of sigaddset() which broke
multiprocessing.  Backport a patch from Python 3.6 to solve this.

(From OE-Core rev: df1874eda517070dab86270ffafd5fe9a307e76b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Ross Burton
27849379a7 python3: remove patch that inadvertently disables xattrs
This patch dates back to the addition of the Python 3 recipe to oe-core, and as
listxattr is never added to supports_follow_symlinks the extended attribute
support will never be enabled.

(From OE-Core rev: 9806e141e85b4e4c38eb7b45e6f2fbc2d2aed29d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Ross Burton
0fa20ac73b python3: remove redundant patch
None of the Python invocations that this changes are actually called, and
there's no need to provide a HOSTPGEN variable when the recipe can just override
PGEN directly.

(From OE-Core rev: 47a8602171428b7ce5d897f7e2c2f26b203b8b63)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Ross Burton
e0fb62596f python3: fix ftplib with TLS 1.3
With OpenSSL 1.1.x TLS 1.3 can be used, so backport a patch from Python 3.6 to
fix the ftplib unit test.

(From OE-Core rev: a31047bec6b7c368674d4620e70e526ac211b936)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-13 07:42:28 +01:00
Ross Burton
dbb2d923de python3: don't use runtime checks to identify float endianism
Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles,
and falls back onto "I don't know" if it can't run code.  This results in
crippled floating point numbers in Python, and the regression tests fail.

Instead of running code, take a macro from autoconf-archive which compiles C
with a special double in which has an ASCII representation, and then greps the
binary to identify the format.

(From OE-Core rev: 1781b87ae0765c1867da2fa8c56bf988b4e84405)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-10 12:13:06 +01:00
Ross Burton
ef38b43dbb python3: trim PGO patch
There's no need to delete the line that removes the profile data, as we're not
using it after the build.  This reduces the size of the patch, making it easier
to maintain.

(From OE-Core rev: 561577add4b935d9bfb7b7dd6abdc3e26c1d9143)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-10 12:13:06 +01:00
Alejandro Enedino Hernandez Samaniego
c05304f1ea python3: fix indentation on create_manifest3
(From OE-Core rev: 76b4596c3782590bd27a7d46c2b64393c3a83944)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-06 10:36:31 +01:00
Alejandro Enedino Hernandez Samaniego
a8a984dde7 python3-manifest: Avoid duplicate modules
There was a bug where modules were being added twice to the
core package, this patches the manifest to reflect the fix on
the create_manifest script.

(From OE-Core rev: 9ddbff824052751b4d0bb30c453ecb5098d728c9)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-06 10:36:31 +01:00
Alejandro Enedino Hernandez Samaniego
f65c898ba6 python3: Clean up create manifest script
This patch intends to clean up the whole create_manifest script/task
for python3.

This is an effort to make the code more human friendly and facilitate
adoption, it not only cleans up the code but it also improves comments,
it should also be easier to upgrade the manifest after each python3
upgrade now, with these fixes the transition to python 3.7 should be
seamless.

It fixes a rather harmless bug where module dependencies were being
added twice to the core package and adds tests and sqlite3-tests
as special packages since we want specific dependencies on those.
It also fixes a bug that happened on a few packages that
contained a directory with the same name as the module itself
e.g. asyncio, where the script avoided checking that module for
dependencies.

Lastly, it improves the output, it errors out if a module is found
on more than one package, which is what usually happens when
python upstream introduces a new file, hence why the current
manifest is unaware of, it is better to exit with an error because
otherwise the user wouldnt know if anything went wrong unless the output
of the script was checked manually.

(From OE-Core rev: 658042073cb58c58ac4db4ff13689d1ffd89b72e)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-06 10:36:31 +01:00
Alejandro Enedino Hernandez Samaniego
d3148222f0 python3: Modify create_manifest to make it versionless
This patch improves the create_manifest script by making it
use PYTHON_MAJMIN instead of hard coded paths containing the
version number when looking at the necessary modules for
every package, the script should now be independent of the
python(3) version on which were working

(From OE-Core rev: b94af33b5ffdd62617cf69fca4d99e927447740a)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-06 10:36:31 +01:00
Alejandro Enedino Hernandez Samaniego
e9b05a3fea python3-manifest: Use PYTHON_MAJMIN instead of hard coded version
Currently the manifest contains hard coded paths with the version number,
e.g. python3.5, this patch changes the paths to use the variable
PYTHON_MAJMIN instead, this should make the python upgrades easier

(From OE-Core rev: a6fab3ca81e3dc6ee226840ed7e9f1cd21ecbc2a)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-06 10:36:31 +01:00
Anuj Mittal
371283df2e python3: enable profile optimized builds
Enable profile guided optimization (pgo) for python3. Enabling pgo in
python is generally as simple as invoking the target profile-opt which:

- builds python binaries with profile instrumentation enabled,
- runs a specific profile task using that python to get the profile
data and,
- feeds the compiler with this profile data and rebuilds python.

This change invokes qemu-user for the second step of running a profile
task using target python. Depending on how long profile task takes to
run, this might add a significant time to compilation (which would be
true for native builds too). The default profile task can be changed by
the users depending on what makes sense for their use case (or can be
left empty). In case qemu-user isn't supported, profile task won't be run.

(From OE-Core rev: 05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16 22:40:28 +01:00
Markus Lehtonen
ef6941fa56 python3: fix depends of python3-tests
Make the tests subpackage depend on all modules as test.regrtest uses most (if
not all) of them.

(From OE-Core rev: f03f3edc211b3e03cf1a6b2655ba664af7fbd12f)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:36 +01:00
Joe Slater
1d708bb185 python3-native: correctly invoke regen-importlib make target
Redefiine regen-all in Makefile to invoke regen-importlib after
building other regen- targets.  Change the recipe to not build it
before regen-all.  This avoids trying to build it multiple times,
which can occasionally fail.

(From OE-Core rev: 72d62c9af07bf34bb8fbb3958742eb592985acc2)

Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04 13:28:05 +01:00
Richard Purdie
7b0074b79b Revert "python3: fix create_manifest to handle pycache folders"
Alejandro asked this be reverted as the patch causes more problems
than it solves.

This reverts commit 5d288d286e.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-07 11:44:50 +01:00
Alejandro Enedino Hernandez Samaniego
b9d51eceac python3: Improve logging capabilities for do_create_manifest
Adds a couple of prints to get a nicer log, and creates a
small summary or report after checking every module, it
makes it more feasible for adoption, easier to debug why
a module ended at a certain package and see how the
manifest was created.

(From OE-Core rev: 4c2af72f51a7bf187615fc0b3a229d25c3e191e9)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-03 23:53:20 +01:00
Alejandro Enedino Hernandez Samaniego
dc24e256a9 python3: Fix do_create_manifest for python3-sqlite3
Some of the sqlite3 files ended up in python3-misc incorrectly,
this is caused becuse we couldnt add the whole ${libdir}/python3/sqlite3
folder on the package because we also have another sqlite3-tests
package that needs to include another folder from that directory.

This patch not only fixes the do_create_manifest script to handle this
situation, but also patches the manifest (created using the script)
which also fixes a hiddn runtime dependency that we wouldn't have seen.

(From OE-Core rev: 3324cb31670f33ffe193e550e3b3da8380b3c8c9)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-31 09:48:42 +01:00
Alejandro Enedino Hernandez Samaniego
5d288d286e python3: fix create_manifest to handle pycache folders
We have a couple of python modules that contain folders themselves,
for that reason they also contain a __pycache__ folder inside those
directories, since we include the whole folder in the manifest, the
pycache directories end up with the files and not the cache files.

This patch catches that and adds the directories to the correct
structure.

(From OE-Core rev: df9401e7e69ce162e257e827d67eb217666e532d)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-31 09:48:42 +01:00
Ross Burton
3a51d2c808 python3: consolidate sqlite3 files
Some of the sqlite3 module was in python3-misc by accident, move the files
into python3-sqlite3 where they belong.

(From OE-Core rev: f06a8d20560b8e93cf875ee58eddda0a976e7b14)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30 00:31:18 +01:00
Jason Wessel
9390f907a9 python3: Fix purelib install and runtime paths
oe-core commit: 45afadf0b6 fixed the pip problem with purelib for
python2, even though the the patch stated it was for python3.  This
patch addresses the purelib problem for python3.

If you install the package python3-pip you will have a pip3 binary
where you can see the problem on the device easily where the modules
install into the incorrect area and are not able to be referenced by
python3 at all.

Example error:
   pip3 install imutils
   pip3 list |grep imutils || echo ERROR no imutils
      ERROR no imutils
   python3 -c 'import imutils'
     Traceback (most recent call last):
       File "<string>", line 1, in <module>
     ImportError: No module named 'imutils'

(From OE-Core rev: 54e0b3bfc132613902418be148a900b10f6d9e38)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-15 06:27:20 -07:00
Ross Burton
a2fdf7fb08 python: refresh patches
The patch tool will apply patches by default with "fuzz", which is where if the
hunk context isn't present but what is there is close enough, it will force the
patch in.

Whilst this is useful when there's just whitespace changes, when applied to
source it is possible for a patch applied with fuzz to produce broken code which
still compiles (see #10450).  This is obviously bad.

We'd like to eventually have do_patch() rejecting any fuzz on these grounds. For
that to be realistic the existing patches with fuzz need to be rebased and
reviewed.

(From OE-Core rev: 8a5c1328c4ea63443a92813c54bd2229c9959ff9)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-11 06:27:00 -07:00
Ross Burton
fe9e4bbee7 python: refresh patches
The patch tool will apply patches by default with "fuzz", which is where if the
hunk context isn't present but what is there is close enough, it will force the
patch in.

Whilst this is useful when there's just whitespace changes, when applied to
source it is possible for a patch applied with fuzz to produce broken code which
still compiles (see #10450).  This is obviously bad.

We'd like to eventually have do_patch() rejecting any fuzz on these grounds. For
that to be realistic the existing patches with fuzz need to be rebased and
reviewed.

(From OE-Core rev: 0b25fcee333e6207a8596d26adfa65fec85c26df)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-09 09:17:04 -08:00
Tomasz Meresiński
c427ed474c python3: Fix python3-logging dependencies
python3-logging depends on python3-netserver (logging/config.py:42)

(From OE-Core rev: ea5f51cf2b885ba32a3e5fe6049141e45606ecf3)

Signed-off-by: Tomasz Meresiński<tomasz.meresinski@comarch.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-16 18:05:39 +00:00
Derek Straka
39e48c8b06 python3: update target and native recipes to 3.5.4
Use the latest 3.5 version until the 3.6 migration is complete

Removed the following upstreamed patches:
  - python3/Fix-29519-weakref-spewing-exceptions-during-interp-f.patch
  - python3/upstream-random-fixes.patch

Rebased the following pathes:
  - python3/0001-cross-compile-support.patch

Regenerated the manifest based on the latest release version

Updated the license checksum for the latest version that updated the
copyright dates

(From OE-Core rev: eb80d0391d7d4e83a61ed8850d936b102be3fa02)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-06 11:06:27 +00:00
Derek Straka
cb61f997c0 python3-manifest: add the _pydecimal files to the numbers package
The _pydecimal files are required to use the numbers package
and downstream packages are currently required to add a RDEPENDS
on python3-misc to avoid an import error

(From OE-Core rev: 4ca2d607a6b6fe8686bf89177287f2b9bae01245)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29 08:49:50 +00:00
Alejandro Hernandez
2afde7e7bd python3: Fix native compilation of gdbm module and manifest
The gdbm module wasnt being built on python3-native showing the following
error during compilation:

Failed to build these modules:
_gdbm

This patch adds the required dependency to fix the compilation problem.

This issue on python3-native caused the manifest creation script to be
unaware of the gdbm library, so this patch also fixes the create_manifest
task for target python, and the manifest file to reflect the changes on
target python as well.

(From OE-Core rev: c47b54aeae5daabb458d6f7118a16257021c1822)

Signed-off-by: Alejandro Hernandez <alejandr@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 22:31:56 +00:00
Tim Orling
5dd24c1753 python*-manifest.json: add dependencies, runpy
* runpy allows running modules/scripts with 'python -m foo'
  - create explicit python2 and 3 packages rather than the
    misc catchall
* python3-setuptools and html.parser RDEPENDS on _markupbase
  - add to python3-core rather than misc catchall
* pip3 RDEPENDS on plistlib, http.client
  - already packaged in python2, add to python3
  - add http/ to -netclient
* "pip3 install" RDEPENDS on encodingds.idna
  - encodings.idna packaged in -core, but missing:
    - stringprep (move from -codecs to -core)
    - unicodedata (move from -codecs to -core)

(From OE-Core rev: 65a85c7db3de8d16ff91a5208a59cc2202d34e5b)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 22:31:56 +00:00