Commit Graph

49 Commits

Author SHA1 Message Date
Ross Burton
fe9e4bbee7 python: refresh patches
The patch tool will apply patches by default with "fuzz", which is where if the
hunk context isn't present but what is there is close enough, it will force the
patch in.

Whilst this is useful when there's just whitespace changes, when applied to
source it is possible for a patch applied with fuzz to produce broken code which
still compiles (see #10450).  This is obviously bad.

We'd like to eventually have do_patch() rejecting any fuzz on these grounds. For
that to be realistic the existing patches with fuzz need to be rebased and
reviewed.

(From OE-Core rev: 0b25fcee333e6207a8596d26adfa65fec85c26df)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-09 09:17:04 -08:00
Tomasz Meresiński
c427ed474c python3: Fix python3-logging dependencies
python3-logging depends on python3-netserver (logging/config.py:42)

(From OE-Core rev: ea5f51cf2b885ba32a3e5fe6049141e45606ecf3)

Signed-off-by: Tomasz Meresiński<tomasz.meresinski@comarch.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-16 18:05:39 +00:00
Derek Straka
39e48c8b06 python3: update target and native recipes to 3.5.4
Use the latest 3.5 version until the 3.6 migration is complete

Removed the following upstreamed patches:
  - python3/Fix-29519-weakref-spewing-exceptions-during-interp-f.patch
  - python3/upstream-random-fixes.patch

Rebased the following pathes:
  - python3/0001-cross-compile-support.patch

Regenerated the manifest based on the latest release version

Updated the license checksum for the latest version that updated the
copyright dates

(From OE-Core rev: eb80d0391d7d4e83a61ed8850d936b102be3fa02)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-06 11:06:27 +00:00
Derek Straka
cb61f997c0 python3-manifest: add the _pydecimal files to the numbers package
The _pydecimal files are required to use the numbers package
and downstream packages are currently required to add a RDEPENDS
on python3-misc to avoid an import error

(From OE-Core rev: 4ca2d607a6b6fe8686bf89177287f2b9bae01245)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29 08:49:50 +00:00
Alejandro Hernandez
2afde7e7bd python3: Fix native compilation of gdbm module and manifest
The gdbm module wasnt being built on python3-native showing the following
error during compilation:

Failed to build these modules:
_gdbm

This patch adds the required dependency to fix the compilation problem.

This issue on python3-native caused the manifest creation script to be
unaware of the gdbm library, so this patch also fixes the create_manifest
task for target python, and the manifest file to reflect the changes on
target python as well.

(From OE-Core rev: c47b54aeae5daabb458d6f7118a16257021c1822)

Signed-off-by: Alejandro Hernandez <alejandr@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 22:31:56 +00:00
Tim Orling
5dd24c1753 python*-manifest.json: add dependencies, runpy
* runpy allows running modules/scripts with 'python -m foo'
  - create explicit python2 and 3 packages rather than the
    misc catchall
* python3-setuptools and html.parser RDEPENDS on _markupbase
  - add to python3-core rather than misc catchall
* pip3 RDEPENDS on plistlib, http.client
  - already packaged in python2, add to python3
  - add http/ to -netclient
* "pip3 install" RDEPENDS on encodingds.idna
  - encodings.idna packaged in -core, but missing:
    - stringprep (move from -codecs to -core)
    - unicodedata (move from -codecs to -core)

(From OE-Core rev: 65a85c7db3de8d16ff91a5208a59cc2202d34e5b)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 22:31:56 +00:00
Alejandro Hernandez
b6777878ff python3: Restructure python3 packaging and replace it with autopackaging
See previous commit (python2 version) for more info, since mostly
everything applies here as well.

Old manifest file had several issues:
 - Its unorganized and hard to read and understand it for an average
   human being.
 - When a new package needs to be added, the user actually has to modify
   the script that creates the manifest, then call the script to create
   a new manifest, and then submit a patch for both the script and the
   manifest, so its a little convoluted.
 - Git complains every single time a patch is submitted to the manifest,
   since it violates some of its guidelines.
 - It changes or may change with every release of python, its impossible
   to know if the required files for a certain package have changed
   (it could have more or less dependencies), the only way of doing so
   would be to install and test them all one by one on separate individual
   images, and even then we wouldnt know if they require less dependencies,
   we would just know if an extra dependency is required since it would
   complain, lets face it, this isnt feasible.
 - The same thing happens for new packages, if someone wants to add a new
   package, its dependencies need to be checked manually one by one.

Features/Fixes:
 - A new manifest format is used (JSON), easy to read and understand.
   This file is parsed by the python recipe and python packages
   read from here are passed directly to bitbake during parsing time.
 - It provides an automatic manifest creation task (explained on previous
   commit), which automagically checks for every package dependencies and
   adds them to the new manifest, hence we will have on each package
   exactly what that package needs to be run, providing finer granularity.
  - Dependencies are also checked automagically for new packages
   (explained on previous commit).

This patch has the same features as the python2 version but it differs
in the following ways:
 - Python3 handles precompiled bytecode files  (*.pyc) differently.
   for this reason and since we are cross compiling, wildcards couldnt be
   avoided on python3 (See PEP #3147 [1]).
   Both the manifest and the manifest creation script handle this
   differently, the manifest for python3 has an extra field for cached
   files, which is how it lets the user install the cached files or not
   via : INCLUDE_PYCS = "1" on their local.conf.
 - Shared libraries nomenclature also changed on python3, so again, we
   use wildcards to deal with this issue ( See PEP #3149 [2]):
 - Fixes python3 manifest, python3-core should be base and everything
   should depend on it, hence several packages were deleted:
   python3-enum, re, gdbm, subprocess, signal, readline.
 - When building python3-native it adds as symlink to it called
   nativepython3, which is then isued by the create_manifest task.
 - Fixes [YOCTO #11513] while were at it.

References:
[1] https://www.python.org/dev/peps/pep-3147/
[2] https://www.python.org/dev/peps/pep-3149/

(From OE-Core rev: 54ac820b8a639950ccb534dcd9d6eaf8b2b736e0)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 22:31:56 +00:00
Markus Lehtonen
ae8d879006 python3: remove two setup.py cross-compile hacks
Remove two unneeded hacks. The first hack ("setup.py: no host headers
libs" patch) is not needed because we use cross-compiler (e.g.
i586-oe-linux-gcc) which has not been configured with any host system
include or library directories, and thus, we don't get any host system
directories when running "gcc -E -v".

The second hack becomes useless after the first hack has been removed
and we get the standard include and lib directories normally from gcc.

(From OE-Core rev: 9e9545ef818adfb57f428db96d4c2ebb0ea65ec6)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-02 11:25:32 +00:00
Juro Bystricky
6fd0e23dbd python3: improve reproducibility
The compiled .pyc files contain time stamp corresponding to the compile time.
This prevents binary reproducibility. This patch allows to achieve binary
reproducibility by overriding the build time stamp by the value
exported via SOURCE_DATE_EPOCH.

Patch by Bernhard M. Wiedemann.

[YOCTO#11241]

(From OE-Core rev: 2175aec10a764bfc925a3fb447547982d0ae662f)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:15 +01:00
Hongxu Jia
365b18dbef python2/python3: fix multiprocessing.BoundedSemaphore not work on qemux86/qemuarm
In upstream, the following commit:
e711cafab1
...
commit e711cafab13efc9c1fe6c5cd75826401445eb585
Author: Benjamin Peterson <benjamin@python.org>
Date:   Wed Jun 11 16:44:04 2008 +0000

    Merged revisions 64104,64117 via svnmerge from
    svn+ssh://pythondev@svn.python.org/python/trunk
...
(see diff in setup.py)
It assigned libraries for multiprocessing module according
the host_platform, but not pass it to Extension.

In glibc, the following commit caused two definition of
sem_getvalue are different.
https://sourceware.org/git/?p=glibc.git;a=commit;h=042e1521c794a945edc43b5bfa7e69ad70420524
(see diff in nptl/sem_getvalue.c for detail)
`__new_sem_getvalue' is the latest sem_getvalue@@GLIBC_2.1
and `__old_sem_getvalue' is to compat the old version
sem_getvalue@GLIBC_2.0.

If not explicitly link to library pthread (-lpthread), it will
load glibc's sem_getvalue randomly at runtime.

Such as build python on linux x86_64 host and run the python
on linux x86_32 target. If not link library pthread, it caused
multiprocessing bounded semaphore could not work correctly.
...
>>> import multiprocessing
>>> pool_sema = multiprocessing.BoundedSemaphore(value=1)
>>> pool_sema.acquire()
True
>>> pool_sema.release()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: semaphore or lock released too many times
...

And the semaphore issue also caused multiprocessing.Queue().put() hung.

(From OE-Core rev: ca1542cdf6b6437a2f3dcdb33ac5216bf841c04a)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-13 09:27:37 +01:00
Mark Asselstine
17d181f059 python3: fix weakref spewing exceptions during interp finalization
When py3 applications are exiting we often see errors similar to the
following:

Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7fcb56b09400>
Traceback (most recent call last):
  File "/usr/lib64/python3.5/weakref.py", line 117, in remove
  TypeError: 'NoneType' object is not callable

After a quick search this was found to be a well reported issue
upstream and had an appropriate fix which is backported here.

(From OE-Core rev: 8aaf09a916a2f66f1a6a79cbddf45390ecefde4f)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 22:51:37 +01:00
Li Zhou
0862df262d python3: Use _sysconfigdata.py to initialize distutils.sysconfig
Porting patch from
<409482251b>
to use _sysconfigdata.py to initialize distutils.sysconfig.

This patch makes that distutils.sysconfig doesn't need config-*/Makefile
in libdir any more. Next we can move it from python3-misc to python3-dev
package.

(From OE-Core rev: 659a80afc5894658f8b82fcd62ebe3562b441db9)

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-29 15:15:20 +01:00
Alexander Kanavin
74684e30ea python: update to 3.5.3
Prior versions of python do not support openssl 1.1; updating to
Python 3.6 on the other hand is a lot more involved, and so should
be done by a specialist/maintainer.

LICENSE checksum change due to copyright years.

Drop upstreamed python3-fix-CVE-2016-1000110.patch

Rebase upstream-random-fixes.patch (taken from
ff558f5aba )

Rebase 0001-Do-not-use-the-shell-version-of-python-config-that-w.patch

Rebase 000-cross-compile.patch

(From OE-Core rev: b7b982a29e5d14c558b5fc25b4dc727810510ade)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:47 +01:00
Alexander Kanavin
3d29214b20 python3: do not hardcode "lib" into site-packages search path
This was not working in multilib or x32 setups and amazingly, was not
noticed until now.

The actual modification is in Lib/site.py, the rest is just devtool moving things around in the patch.

(From OE-Core rev: f60d261b682f1526fb7a754c425300954ef85042)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:43:13 +01:00
Jose Lamego
4ad1148e68 python-3.3-multilib.patch: Fixes getpath on multilib configurations
When using multilib configurations either on arm/arm64 and x86/x86-64
python3 failed to execute due to a failure when looking for its
platform independent and dependent libraries.

This patch fixes this issue by assigning lib_python to the appropriate
macro.

[YOCTO #10812]

(From OE-Core rev: 9e99897f17d9c62ca5da208751d6560fc98927b6)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Jose Lamego <jose.a.lamego@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Jussi Kukkonen
82d817826c python3: Remove unused patches
CVE-2016-5636.patch and use_packed_importlib.patch were removed from
SRC_URI as handled in upstream in b192bc02bb.

(From OE-Core rev: 71c539483350a4b45fb5d397d29b6ebcfc863e2f)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16 22:11:32 +00:00
Richard Purdie
a0336965aa python3: Add upstream random.c fixes for recent glibc
python3 fails to work with recent glibc versions on older hosts, giving
errors like:

Fatal Python error: getentropy() failed
Aborted

This breaks buildtools-tarball and hence eSDK. This patch backports the
changes to random.c from upstream that address the problem.

(From OE-Core rev: 126b2c47b1806b53fbd9a4706bc48bc7c4efd3be)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Dominic Sacré
3f2ab3f7cc python3: Build and package precompiled modules
Remove the patch that was applied in the python3 and python3-native
recipes to skip compilation of python modules.

Modify generate-manifest-3.5.py to match '__pycache__' directories in
FILES_*.
This is necessary because Python3 puts .pyc files in '__pycache__'
subdirectories one level below the corresponding .py files, whereas in
Python2 they used to be right next to the sources.

This change significantly reduces the startup overhead of Python3
scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s
before, and 0.19s after.

(From OE-Core rev: bb4d689769703177dbb0df0935e15016b879f42b)

Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Mingli Yu
fd4391d8ba python: fix CVE-2016-1000110
Backport patch to fix CVE-2016-1000110 from python upstream:
for python2.7
https://hg.python.org/cpython/rev/ba915d561667/
for python3
https://hg.python.org/cpython/rev/a0ac52ed8f79

(From OE-Core rev: 1dd22b9d35983f35c481a1fcf67425aa0fd07a5b)

Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Alejandro Hernandez
3ab991bee2 python3-native: Upgrade from 3.5.1 to 3.5.2
LICENSE did not change, only dates were changed

Rebases:
 - 000-cross-compile.patch
 - python-3.3-multilib.patch

(From OE-Core rev: 0a3a4047e779c8bff2b5e2bfa37b7ab119d08d4b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-10 10:46:30 +01:00
Armin Kuster
d1fdd0ad55 python3: Security fix CVE-2016-5636
Affects python3 < 3.5.1
Base Score (4.4) Medium

(From OE-Core rev: a7cbd6805febadaad60d1e05899e10e9a8a36c26)

Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:52 +01:00
Ross Burton
e49d337e84 meta: update patch metadata
Enforce the correct tag names across all of oe-core for consistency.

(From OE-Core rev: 606a43dc38a00cc243f933722db657aea4129f8e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Alexander Kanavin
f5d36b1c56 python3: drop 110-enable-zlib.patch
With the patch, the zlib extension wasn't built at all.
Without the patch, it gets build just fine.

(From OE-Core rev: 59870a852cf88ecef094db0a2adec168a8997626)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-03 13:13:26 +01:00
Alexander Kanavin
083d0aa27f python3: add = to -L linking option only when the path is absolute
Previously it was added also when the path was relative and not
prefixed with ./, which was causing issues with building numpy.

(From OE-Core rev: 3e171c89e929a09e4d511a8f235dd90b7cf0d463)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:42 +01:00
Alexander Kanavin
4977a0743b python-native, python3-native: remove the use of exported HOST_SYS and BUILD_SYS variables
The code that utilized them was superseded by the code (in the same patch!)
that is utilizing STAGING_LIBDIR/STAGING_INCDIR, and wasn't correct in the
first place as HOST_SYS is not necessarily the same as the sysroot directory
name.

(From OE-Core rev: 8834e81a38c24a066bb4fefa93da61011d0db244)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Martin Jansa
258cfa891e python(3): Disable tkinter
* restore changes from:
  http://git.openembedded.org/openembedded-core/commit/?id=53ae544cfdac22c82af452b8c7ebe6664296bd9b
* which were shamelessly removed in upgrade to 2.7.9:
  http://git.openembedded.org/openembedded-core/commit/?id=d4ad95f0d5f08891637c644e85b09da9c4585059
  and then spread to python3 as well
* fixes following issues reported by test-dependencies
  WARN: python3: python3-tkinter rdepends on glibc, but it isn't a build dependency?
  WARN: python3: python3-tkinter rdepends on libpython3, but it isn't a build dependency?
  WARN: python3: python3-tkinter rdepends on tcl-lib, but it isn't a build dependency?

  WARN: python: python-tkinter rdepends on glibc, but it isn't a build dependency?
  WARN: python: python-tkinter rdepends on libpython2, but it isn't a build dependency?
  WARN: python: python-tkinter rdepends on tcl-lib, but it isn't a build dependency?

  and following QA warnings in normal builds:
  python-2.7.11: python-tkinter rdepends on tcl-lib, but it isn't a build dependency, missing tcl in DEPENDS or PACKAGECONFIG? [build-deps]
  python-2.7.11: /usr/lib/python2.7/lib-dynload/_tkinter.so contained in package python-tkinter requires libtk8.6.so, but no providers found in RDEPENDS_python-tkinter? [file-rdeps]
  python3-3.5.1: python3-tkinter rdepends on tcl-lib, but it isn't a build dependency, missing tcl in DEPENDS or PACKAGECONFIG? [build-deps]
  python3-3.5.1: /usr/lib/python3.5/lib-dynload/_tkinter.cpython-35m-arm-linux-gnueabi.so contained in package python3-tkinter requires libtk8.6.so, but no providers found in RDEPENDS_python3-tkinter? [file-rdeps]

(From OE-Core rev: 8cc5e36cd7978186d5ea1fe8af2f2bd6b3df8885)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:14 +00:00
Hongxu Jia
3d45853eef python3: fix do_configure check platform triplet error
For p1022ds bsp, the MULTIARCH is powerpc-linux-gnuspev1 and
python3 did not recognize the extra 'v1' which caused python3
configure error for the platform triplet.

Q:Why Python3 check platform triplet?

A:Under Linux, GNU/KFreeBSD and the Hurd, C extensions now include
the architecture triplet in the extension name, to make it easy to
test builds for different ABIs in the same working tree.

vim config.log
...
EXT_SUFFIX='.cpython-35m-powerpc-linux-gnuspe.so'
...

Here is the generated C extensions which take platform triplet into account.
...
|image/usr/lib/python3.5/lib-dynload/_datetime.cpython-35m-powerpc-linux-gnuspe.so
...

https://bugs.python.org/issue22980
https://www.python.org/dev/peps/pep-3149/
https://bugs.python.org/review/22980/patch/14593/54808

[YOCTO #9226]

(From OE-Core rev: cda0ef61d37357fed1daa22f6a59ef9f906fcada)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12 22:11:46 +00:00
Alexander Kanavin
142bad3615 python3: fix patching get_python_lib() in distutils/sysconfig.py
Previous, two things were wrong:
1) lib_basename was set from STAGING_LIBDIR only if prefix parameter was empty or missing
2) if prefix was not empty, lib_basename reverted to sys.lib, even if STAGING_LIBDIR
should've overriden it

(From OE-Core rev: 28d29004aa7d17794216d7df55afc308b1f0e806)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:28 +00:00
Alexander Kanavin
50d07e923e python3-native: use the previous version of python-config script
In python 3.4 python-config was rewritten in shell, ironically
to support cross-compilation:
https://bugs.python.org/issue16235

This new shell version is broken in several ways, and doesn't
have our oe-specific tweaks. Let's revert to the old script,
which is still provided.

(From OE-Core rev: b0d714c7f831828a5e09381fe36e3f859c16b2d9)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:28 +00:00
Ross Burton
3adb8d5cde python3: remove optimize by default patch
Python 3 resurrected the default to optimised patch, on the rationale that
embedded systems want all the performance they can get.  Unfortunately the only
impact Python "optimisation" has is to remove all asserts which can be actively
harmful, so remove this patch.

(From OE-Core rev: 1e43b836106ed13cde294e2c565eaea6fe5efca7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07 17:29:59 +00:00
Alejandro Hernandez
113e1365a8 python3: Minor upgrade 3.5.0 -> 3.5.1
Rebased:
- 000-cross-compile.patch

(From OE-Core rev: f1d9f7b2ea930fec3198264a2d1d04088efc8f3c)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Robert Yang
29d273fdba python3: fix installed-vs-shipped when 64bit + multilib
Fixed:
python3-3.5.0: python3: Files/directories were installed but not shipped in any package:
  /usr/lib
  /usr/lib/python3.5
  /usr/lib/python3.5/config-3.5m
  /usr/lib/python3.5/config-3.5m/python.o
  /usr/lib/python3.5/config-3.5m/Setup.local
  /usr/lib/python3.5/config-3.5m/Setup
  /usr/lib/python3.5/config-3.5m/python-config.py
  /usr/lib/python3.5/config-3.5m/install-sh
  /usr/lib/python3.5/config-3.5m/config.c.in
  /usr/lib/python3.5/config-3.5m/makesetup
  /usr/lib/python3.5/config-3.5m/Setup.config
  /usr/lib/python3.5/config-3.5m/config.c
  /usr/lib/python3.5/config-3.5m/Makefile
  /usr/lib/python3.5/config-3.5m/libpython3.5m.a
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
python3: 14 installed and not shipped files. [installed-vs-shipped]

(From OE-Core rev: 8a96dbe5dd44e1c80aef7b8efe5820dee96a5bd7)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Alejandro Hernandez
2268a702f1 python3: Upgrade from 3.4.3 to 3.5
python3-native_3.4.3.bb -> python3-native_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

python3_3.4.3.bb -> python3_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

New:
- use_packed_importlib.patch: Fixes importlib on cross-compile environments

Rebased:
- Manifest
- 000-cross-compile.patch
- 020-dont-compile-python-files.patch
- 04-default-is-optimized.patch
- python-3.3-multilib.patch
- distutils3-base.bbclass
- distutils3-native-base.bbclass
- python3native.bbclass

Upstream:
- makerace.patch

Misc:
- pip2 is handled as default on major distros,
modified python3-pip to leave /usr/bin/pip available for pip2
- Fixed importing pip3 from python3 interpreter

(From OE-Core rev: 701ec1977ced1bb08461e6de98b4f63d21cba8a6)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:06 +00:00
Ross Burton
1c914a844b meta: Fix Upstream-Status statements
Fix a variety of problems such as typos, bad punctuations, or incorrect
Upstream-Status values.

(From OE-Core rev: bd220fe6ce8c3a0805f13a14706d3130ea872604)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12 23:01:53 +01:00
Dominic Sacré
10196e3465 python3: delete patches that don't get applied anywhere
These patches are not referenced in any of the Python recipes.

(From OE-Core rev: 05d18a1d2630fbe81d8218c3722481d31bf0892f)

Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:18 +01:00
Dominic Sacré
3c53a2bbd7 python3: detect libraries in STAGING_LIBDIR
Patch setup.py so that the detect_modules() function looks for required
libraries and headers in STAGING_LIBDIR / STAGING_INCDIR.

Without this patch, several extension modules are not built, even though
their dependencies are present in the compiler's search paths.
The result is the following warning, and ultimately incomplete packages:

| The necessary bits to build these optional modules were not found:
| _bz2                  _curses_panel         _dbm
| _gdbm                 _lzma                 _sqlite3
| nis                   readline              zlib
| To find the necessary bits, look in setup.py in detect_modules() for the module's name.

With the Python 3.3 version of the recipe, at least some of these modules
were built, because STAGING_LIBDIR happened to be in the search path
by coincidence. Due to changes to distutils in Python 3.4, this is no
longer the case.

A previous patch that only affects the search paths for SSL is dropped, as
this one is a more complete fix for [YOCTO #7768].

Part of the fix for [YOCTO #8048].

(From OE-Core rev: 540a223fc0fb6b07dab3316de6236924bd97613c)

Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:17 +01:00
Alejandro Hernandez
ae67f744a5 python3-io: Fixes compilation of python3 ssl module
During python3 compilation, the module ssl was being skippped due to
the fact that the compilation script couldnt find the required files,
this patch fixes setup.py so it looks for the files in the correct
directory, hence fixing its compilation and installation.

[YOCTO #7768]

(From OE-Core rev: 4cffb16b0edc353d4a3287ca59ba02640f605d2b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-08 17:34:04 +01:00
Alejandro Hernandez
0be72c2fb5 python3-native: Fix pip install issue due to unclean build directory
When installing python3-native sometimes pips default build
directory (which is on the host and is user dependant) is left unclean,
due to this, when python3-core is being installed it tries to use
the same directory producing an error, this explicitly removes
what the previous installation might have left behind, fixing the issue.

(From OE-Core rev: c27a5a9ba649e7fcf681f6ac4575442e252fd29b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07 13:40:33 +01:00
Alejandro Hernandez
19149e67e8 python3: Upgrade from 3.3.3 to 3.4.2
Modifies:
python3-native_3.3.3.bb -> python3-native_3.4.2.bb: Updates checksums, version, patches
and SRC_URIs to use xz format, LICENSE did not change, dates were updated, adds fix for
ctypes/libffi needed for pip <- required since Python 3.4 (see PEP #453).
python3_3.3.3.bb -> python3_3.4.2.bb: Updates checksums, version, patches and SRC_URIs
to use xz format, adds fix for ctypes/libffi, fixes oe_multilib_header() call,
adds python3-core as RDEPENDS to python3-misc to avoid QA issue.
generate-manifest-3.4.py - > generate-manifest-3.4.py: fixes reprlib,
adds _sitebuiltins and _collections_abc <- formerly part of collections, separated since 3.4
python-3.3-manifest.inc -> python-3.4-manifest.inc

Rebases:
12-distutils-prefix-is-inside-staging-area.patch
python-3.3-multilib.patch
04-default-is-optimized.patch
avoid-ncursesw-include-path.patch

Deletes (fixed upstream):
fix-ast.h-dependency.patch
python3-fix-build-error-with-Readline-6.3.patch
06-ctypes-libffi-fix-configure.patch
remove_sqlite_rpath.patch

[YOCTO #6857]

(From OE-Core rev: 45290ff4d1902bd0ab5377568f969fbb660c0c87)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07 13:40:33 +01:00
Robert Yang
31e6dda52d python3: remove sys_platform_is_now_always_linux2.patch
It is aready in the source.

(From OE-Core rev: 99f9df3add049ec18dbcd604646a67dc59b3db16)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-30 23:04:12 +01:00
Robert Yang
b2f8367dff python3: fix for cross compiling
Fixed:
* python3 has introduced _PYTHON_PROJECT_BASE which is used for separate
  B and S, but it doesn't work when compile Modules, the target python3 runs
  python3-native's sysconfig to get srcdir which is the native's, there
  would be errors when native's srcdir has been removed, add
  _PYTHON_PROJECT_SRC to fix the problem.

* Check cross_compiling when get FLAGS

(From OE-Core rev: c33639963491f00f55b80299922895fe68b0637d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:27 +00:00
Jackie Huang
ce8c56f838 python3: several fixes for cross compiling
* Add a patch to use CROSSPYTHONPATH as PYTHONPATH for
  PYTHON_FOR_BUILD, otherwise CROSSPYTHONPATH is never used,
  and it use the path in target builds to find libraries.

* Add a patch to avoid finding host headers and libs

* Fix a typo: s/python-native3/python3-native/

(From OE-Core rev: d3d00163671bda5395c9046c1109f711772e4ed9)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25 13:03:27 +00:00
Roxana
0ed5a13f5f python3: fix builtins imports
The intent of this code is to generate things like -L=/usr/lib/foo so
for paths which start with "/" we should appent the "=". We should not do this
for ".". On some recent versions of binutils or patchsets on some recent
distros this appears to work differently and causes build failures, for example
on Ubuntu 14.04 LTS.

The simple fix is to check for "." as the path prefix.

[YOCTO #6467].

(From OE-Core rev: c36d459f0d40bdbd3ba809835e0475e8992bc778)

Signed-off-by: Roxana <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-30 16:36:44 +01:00
Tyler Hall
28798f1963 python3: Revert python-config to distutils.sysconfig
The newer sysconfig module shares some code with distutils.sysconfig,
but the same modifications as in

12-distutils-prefix-is-inside-staging-area.patch

makes distutils.sysconfig affect the native runtime as well as cross
building.  Use the old, patched implementation which returns paths in
the staging directory and for the target, as appropriate.

This change reverts this upstream patch
http://hg.python.org/cpython/diff/712970b019f7/Misc/python-config.in

(From OE-Core rev: 7b2ffd68ae8235dcc3ddff9cbe8525e61f3b3d28)

Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 13:00:32 +01:00
Richard Purdie
96e488b76a python3/gcc/autoconf: Fix Upstream-Status in some patches I authored
(From OE-Core rev: 337798fa5c0a1d1e745a143f6a9f398b07f0628f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-07 15:14:34 +01:00
Chong Lu
b996f22f8b python: fix build error with Readline 6.3
Backport two patches from upstream:

use new readline function types (closes #20374)
Issue #20374: Avoid compiler warnings when compiling readline with libedit.

[YOCTO #6107]

(From OE-Core rev: a6b91ae7dec2edebc0eaea0592c42b1c455ad4d7)

Signed-off-by: Chong Lu <Chong.Lu@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 11:53:51 +01:00
Richard Purdie
5436de5274 python3: Fix make race
This fixes a race which was showing up on the autobuilder (see patch header).

(From OE-Core rev: 9d796611afc42bea5ad9cab47ba4fb5ac0cb5ddd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:01:15 +00:00
Khem Raj
8a33c5dd3b python3: Fix race condition at high parallelism factor
Backport a patch to fix race condition as reported here

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5884

[YOCTO #5884]

(From OE-Core rev: 209a019b12f2941f8aefac9f192e9cdf691196e3)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:01:12 +00:00
Khem Raj
1c8869bca4 python3: Add target and native recipes
3.3.3 is latest bugfix release in 3.3 series

License text is changed to reflect year and prev releases

+++ ../Python-3.3.2/LICENSE     2013-05-15 09:32:54.000000000 -0700
@@ -74,7 +74,9 @@
     3.2.1           3.2         2011        PSF         yes
     3.2.2           3.2.1       2011        PSF         yes
     3.2.3           3.2.2       2012        PSF         yes
+    3.2.4           3.2.3       2013        PSF         yes
     3.3.0           3.2         2012        PSF         yes
+    3.3.1           3.3.0       2013        PSF         yes

 Footnotes:

@@ -110,8 +112,8 @@
 distribute, and otherwise use Python alone or in any derivative
version,
 provided, however, that PSF's License Agreement and PSF's notice of
copyright,
 i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008,
2009, 2010,
-2011, 2012 Python Software Foundation; All Rights Reserved" are
retained in Python
-alone or in any derivative version prepared by Licensee.
+2011, 2012, 2013 Python Software Foundation; All Rights Reserved" are
retained
+in Python alone or in any derivative version prepared by Licensee.

We use native python in disutils bbclass and this needs
to peek into target sysroot when building plugins in cross
environment. Otherwise anything that inherits distutils3.bbclass
will not build.

Fix host include contamination issue

Let compiler append sysroot to include path if it can

Fix the compiler invocation and linker flags when cross compiling

The details are in patch header. This should fix the QA errors about
host contamination

Add virtual/libintl to dep list

There is use of libintl.h but we dont
express the dependency

Add lzma dependency for python3-misc

Fixes

and from test-dependencies.sh run there is undeterministic lzma
dependency:

WARN: packages/armv5te-oe-linux-gnueabi/python3/python3-misc/latest lost
dependency on  liblzma

Fix staging warnings due to duplicate install

WARNING: The recipe python3 is trying to install files into a shared
area when those files already exist. Those files and their manifest
location are:

builds/world/tmp/sysroots/qemux86-64/usr/include/python3.3m/pyconfig.h
builds/world/tmp/sysroots/qemux86-64/usr/lib/libpython3.3m.so.1.0
builds/world/tmp/sysroots/qemux86-64/usr/lib/libpython3.3m.so
builds/world/tmp/sysroots/qemux86-64/usr/lib/python3.3/config/Makefile

Let compiler search includes relative to sysroot

This makes it cross compilable and assumption on hardcoded
paths is fixed.

Errors like below would appear if ncursesw is installed on build host

cc1: warning: include location "/usr/include/ncursesw" is unsafe for
cross-compilation [-Wpoison-system-directories]

(From OE-Core rev: 185fcfc3286b355394ae89300f22b8cb7aaa43c1)

Signed-off-by: Khem Raj <raj.khem@gmail.com>

Fix linking not happening issue in modules

We need to tweak CONFIGURE_LDFLAGS for python3 instead of LDFLAGS
directly

Add second compile stage which uses original makefile

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-02 22:37:39 +00:00