Commit Graph

257 Commits

Author SHA1 Message Date
Holger Freyther
83ec5eaed4 bitbake: [parser] Cache parsed .inc and .bbclass files for a parse speedup
Have a growing dict with .inc and .bbclass'es. This avoids to reparse
files we have already seen.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:54 +00:00
Holger Freyther
c011d42eda bitbake: [parser] prepare to cache some .bbcclass and .inc files
Our parser is shit but instead to replace it now we will see
how long we can drive the wave by caching parsed files. This
will not go through the feeder again but we can just reevaluate
the StatementGroup.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:53 +00:00
Holger Freyther
793c88dd92 bitbake: [parser] Remove the "data" from feeder, evaluate after parsing a file
Evaluate the statements after having parsed one file. This is
referred to as "entwirren" and we can remove the direct evaluation
and postpone a bit, in the future we can use a cached copy instead
of parsing the original.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:51 +00:00
Chris Larson
3eb2e6cf02 bitbake: [parse] Move vars_from_file from bb.parse.BBHandler into bb.parse.
(Bitbake rev: fda0707d772e0964a0185d4ec4d016522f6972f3)

Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:50 +00:00
Chris Larson
6f52a1521e bitbake: [parse] Don't use relative import of parse_py.
(Bitbake rev: 0a2bb3dd790e3e40867195f14f4e174f98f8a47c)

Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:49 +00:00
Holger Freyther
3f1ca0a086 bitbake: [parse] Move the last method to the AST...
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:48 +00:00
Holger Freyther
baf2395900 bitbake: [parser] Move more methods to the ast scheme
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:48 +00:00
Holger Freyther
d84b7c5555 bitbake: [parser] Move one more item to the ast
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:47 +00:00
Holger Freyther
a5bb5d625e bitbake: [parser] Move methodflags over to the ast
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:46 +00:00
Richard Purdie
7cfe3a5e15 bitbake: [parser] Build fix
(Bitbake rev: 9c97696f37499b4d0ec5c034c51e4cf6bc425ba2)

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:45 +00:00
Holger Freyther
8eaaf663ba bitbake: [parser] Move more statements over the two phase AST
Create the data first, then evaluate on the data dict

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:44 +00:00
Holger Freyther
d2bf3f00ea bitbake: [parse] Change handling include through the AstNode
Disable the recursively evaluating the statement for now
as it is causing problems.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:43 +00:00
Holger Freyther
8072cf0846 bitbake: [parser] Call eval and remove non ast code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:42 +00:00
Holger Freyther
118161f46b bitbake: [parser] Call eval immeditaley to test this code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:41 +00:00
Holger Freyther
a20105868f bitbake: [parser] Prepare to cease out getFunc
getFunc is now a method of the data node, hopefully we can kill the other
version soon.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:40 +00:00
Richard Purdie
269d4d58a9 bitbake: [parser] Preserve include vs. require behaviour
(Bitbake rev: 6073a5b8e4ca8af8e1a8e0234fad7b08baf76c99)

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:20 +00:00
Holger Freyther
71a19386a3 bitbake: [parser] Firs set of AST nodes with construction and eval
First set of ConfHandling with AST nodes. The include can
use a speed up and things might need to be migrated... into
this class.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:07:07 +00:00
Holger Freyther
df8f7706a6 bitbake: [parser] Add documentation about usage of this code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:06:59 +00:00
Holger Freyther
4320386deb bitbake: [parser] Move the finalise into the ast as well
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:06:55 +00:00
Holger Freyther
4b25b519ae bitbake: [parser] Cary a Statement Node through the parsing
When parsing we will collect a number of statements
that can be evaluated...The plan is to be evaluate
things twice (old+new) and then compare the result,
it should be the same.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:06:50 +00:00
Holger Freyther
5bac3403d7 bitbake: [parser] Move evaluating into the ast class...
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:06:36 +00:00
Bernhard Reutner-Fischer
e9d8dd2abf bitbake: providers.py: sort eligible providers by DEFAULT_PREFERENCE
(Bitbake rev: ee9afccf33b220a21b74fab279925eeb4771249b)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:03:50 +00:00
Bernhard Reutner-Fischer
de7c939a7f bitbake: runqueue.py: improve printing dependent tasks
Print names instead of Task-IDs (and not mentioning they're task ids).

Previously we printed e.g.:
Dependency loop #1 found:
  Task 89 (/there/src/oe/openembedded/recipes/busybox/busybox-native_1.15.2.bb, do_configure) (depends: Set([88, 282, 92, 87]))

Now we say
Dependency loop #1 found:
  Task 89 (/there/src/oe/openembedded/recipes/busybox/busybox-native_1.15.2.bb, do_configure) (dependent Task-IDs ['busybox-native, do_patch', 'update-rc.d, do_populate_staging', 'busybox-native, do_populate_staging', 'shasum-native.bb, do_populate_staging', 'busybox-native, do_unpack'])

(Bitbake rev: 00eaf76fdc32eb515995b47dfa69eb90ca904b37)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:03:47 +00:00
Bernhard Reutner-Fischer
e39d89e934 bitbake: taskdata.py: fix typo in debug message
(Bitbake rev: 457fb59f10ed59269b8bda3ee53bbeded5d33eb5)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:03:07 +00:00
Richard Purdie
62c103ce0e bitbake: utils.py: Don't error if an exported environment variable is unset (i.e. None)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-12 13:50:19 +00:00
Khem Raj
99b4b10dc1 bitbake: BBHandler: pass lineno and fn to handleMethod
* lineno and fn are needed in handleMethod
  to restore the functionality as it was before.

(Bitbake rev: ac6792045959cfee56279c1c4597521e990848e7)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:43:06 +00:00
Martin Jansa
d5519d4ad9 bitbake: parse_py/ConfHandler.py: missing colon after else
(Bitbake rev: d520ae0764016906ff61ec33b14eabc908aa8408)

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:38:33 +00:00
Richard Purdie
7a4579e7cb bitbake: ConfHandler: Fix require vs. include behaviour
(Bitbake rev: 824a032672c8673cae28e492aa916a4750b94efa)

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:38:00 +00:00
Holger Freyther
fb918a7582 bitbake: [parser] Move more stuff out the feeder
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:37:43 +00:00
Holger Freyther
b045ab3222 bitbake: [parse] Move inherit handling to method
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:32:28 +00:00
Holger Freyther
7477c0d89a bitbake: [parsers] Move out more functions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:32:14 +00:00
Holger Freyther
e3c9a340dc bitbake: [parser] Move more stuff out to separate methods...
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:32:00 +00:00
Holger Freyther
169e719456 bitbake: [parser] Move the handling of a method to a function
We want to convert this into a proper AST. So move all
such operations to methods... Later change them to generate
a node... and create that node from here.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:31:48 +00:00
Holger Freyther
913e78898e bitbake: [parse] Unify opening a file...
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:31:33 +00:00
Holger Freyther
242a03607d bitbake: [parser] Kill obtain/localpath from the parser
With obtain it was possible to use an existing fetcher to
download a bb or config file. In practive no one has used it
and it was likely broken in regard to depends_cache... Remove
it for now, simplfiy the code.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:31:19 +00:00
Theodore A. Roth
bf6a4fd658 bitbake: usermanual.xml: Add section for ?= var setting.
(Bitbake rev: df8309c11a43a9488ca25ec114a2dd3d49b041ff)

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-10 16:25:35 +00:00
Joshua Lock
52cbb07423 bitbake: fix invalid method and logic
The previous bitbake change included an invalid method call and inverted logic.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-05 12:41:08 +00:00
Joshua Lock
ec92133d6f bitbake: Test premirrors and mirrors in checkstatus()
checkstatus() is used to ensure we can fetch a copy of each file, so it makes
sense to also test PREMIRRORS and MIRRORS in the method.

This patch adds calls to try_mirrors() to the Fetch.checkstatus() method and
changes the try_mirrors() method to take a check argument, which is False by
default. When check is True try_mirrors() will call a fetchers checkstatus()
with the replaced uri.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-05 12:19:05 +00:00
Joshua Lock
c9fd126f64 bitbake: Fix try_mirrors() logic, was only trying the first mirror.
The logic in try_mirrors() was buggy such that only the first entry in the
mirrors list was tried.

This patch fixes this. Thanks to Richard for the review.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 14:08:34 +00:00
Joshua Lock
afe12428a9 bitbake: if PREMIRRORS set test for local file in FetchData.setup_localpath
When we are using PREMIRRORS it's possible a mirror in the local namespace
(some filesystem path, i.e. an NFS share) provides read-only files.
This is a perfectly valid scenario so this patch fixes bitbake so that for
such a scenario locapath is set to the files path rather than some child
of DL_DIR.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Joshua Lock
b571168ac7 bitbake: Enhance the fetchers' support for local mirrors
Modify the try_mirrors() function to return the localpath of the fetched file
and update the data dictionary to reflect this.
Secondly the metadata files, lock and md5, should always be stored relative to
the ${DL_DIR} as it is possible that the localpath is a read-only directory,
for example in the scenario where there is a read-only file:// mirror.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Joshua Lock
0737552c1d bitbake: unify mirror support and make it independant of the fetcher
This patch serves two purposes. Firstly it unifies the concept of mirrors into
PREMIRRORS and MIRRORS. PREMIRRORS are tried before the SRC_URI defined in the
recipe whereas MIRRORS are tried only if that fails.
The tarball stash was conceptually inline with a PREMIRROR only with special
handling within the wget fetcher and therefore only worked with certain
fetch types.
Secondly the patch removes the need for individual fetch implementations to
worry about mirror handling.

With this patch, the base fetch implementation will first try to use a
PREMIRROR to fetch the desired object, if this fails the native fetch method
for the object will be tried and if this fails will try to fetch a copy from
one of the MIRRORS.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Richard Purdie
fdf134abfb bitbake/build.py: No need for this gross hack now
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-27 12:40:02 +00:00
Richard Purdie
b9d958c186 bitbake: Don't import xmlrpc server unless needed
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-22 16:15:50 +00:00
Richard Purdie
aedf7d3205 bin/bitbake: Add import server
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-22 09:00:49 +00:00
Richard Purdie
7e8025459e bitbake/cooker: Drop unneeded import
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-22 00:04:18 +00:00
Richard Purdie
37f8e24ce3 bitbake: Fix UI failure exception handling with python 2.6
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 23:49:50 +00:00
Richard Purdie
5ebad0d500 knotty: Improve task failure message handling using the uihelper code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 23:49:17 +00:00
Richard Purdie
5c62833766 runqueue.py: Use fcntl to make the worker pipes non-blocking
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 23:46:20 +00:00
Richard Purdie
fd42ffa273 bitbake/event: If the server disappears, we just exit too
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 16:00:12 +00:00