Comments (17)
Imported from trac issue 17. Created by jphickey on 2015-02-09T11:06:56, last modified: 2015-11-20T16:22:16
from osal.
Trac comment by jphickey on 2015-02-09 12:23:41:
== More info ==
In addition to simply making the OSAL tests cleaner and more consistent, this actually has real benefits to the build management as well. Since CFE and OSAL are separate repos (which is logical), it also means that any change to an OSAL header file may need a corresponding change to the CFE code. For most application code this is not an issue; as long as the change is compatible, the compiler does the "right thing" and all code will still work.
The real build problem, however, occurs in the "ut_osapi_stubs.c" file within the CFE build. Unlike normal app code, this file actually //implements// stub versions of OSAL calls, so the compiler is very picky about the prototypes being exactly identical here. To summarize the problem:
- The prototypes are in a ".h" file supplied by OSAL and checked out from the OSAL repository as one commit.
- The implementation is in a ".c" file supplied by CFE and checked out from the CFE repository as a //different// commit.
This means a lot of potential pitfalls going forward. as it becomes really easy to end up with incompatible sources as change sets are mixed and matched. There is a corresponding ticket on the CFE side for the same issue: [https://babelfish.arc.nasa.gov/trac/cfs_cfe/ticket/32]
== Plan ==
-
Migrate "ut_osapi_stubs.c" to the OSAL repo so that the implementation will always change with the prototypes in the same change set. (It will likely need to coexist in both places for a transitional period)
-
Migrate implementation of common UT functions such as UT_Report(), UT_SetRtnCode() etc to a separate UT library.
-
Migrate implementation of the UT_Text() output function to the OSAL UT BSP- this can be the existing pc-linux-ut bsp or a target-specific UT bsp. Note that this implementation only needs to be in the "ut" BSP, not the normal runtime.
-
Provide the UT library + BSP as a platform for the CFE unit tests to execute.
Additional benefits: moving code into the OSAL BSP should also remove the need for a lot of the {{{#ifdef / #endif}}} blocks currently in the CFE unit tests, namely the alternate implementation for the ARINC653 environment which does not use stdio printf(). This will clean all of that up by putting the special code into the BSP which is unique to that platform. The CFE unit tests could then be easily built and run on any plaftorm with an OSAL "ut" bsp implemented. The current #ifdefs (e.g. {{{#ifdef CFE_LINUX}}}) will not scale to many platforms - it appears that only Linux and the ARINC653 environment are supported.
from osal.
Trac comment by jphickey on 2015-02-09 12:26:43:
Any comments/suggestions on the above ideas are welcome. This is not something I've implemented (yet) but as long as we have some agreement on the benefits here I would be happy to implement this - should not take too long.
from osal.
Trac comment by glimes on 2015-02-10 12:04:03:
Joe, I've nosed into this territory a couple of times.
The transition plan looks sensible.
I'd like to see a clearer demarcation between API calls manipulating
stubs and mock systems, from the API calls that report the results
of unit testing: the former are intrinsically specific to the system
being tested, while the latter may be kept stable from project to
project, which means we could deploy a common UT reporting
library across the board.
I have been able to integrate CFE unit tests into a Bamboo build,
so moving OSAL toward that API sounds like a good idea to me.
Special constraints are that doing this requires organizing the
supporting text for a failed condition along with the indication
that the condition failed, all presented as a single blob of text
into the output; this also has to identify the associated test case.
''Side note'': Bamboo notes a testsuite name and testcase name
for each failure, and tracks everything based on the name tuple,
so the vertical depth of the heirarchy available for the entire
collection of tests run by a build is uncomfortable, but seems
to be acceptable for the CFE unit tests. The actual Bamboo
presentation of those results is over on our ITAR server, and
linking from here would be futile, rather than being a helpful
example of an implementation. I'll figure out some way to pull
this example out into the light.
from osal.
Trac comment by jphickey on 2015-02-10 13:06:38:
Thanks for the comments. I think this will achieve some of what you are looking for in terms of isolation between the pass/fail reporting calls and the stub functions.
I am putting together a header file and C file that reflect the ideas I'm thinking of here. None of the UT code has been modified to use it yet, but it could act as a preview of sorts to show how the API would look/work.
This mostly looks like the existing API so there shouldn't be any major concerns, the major difference being that it shifts some logic into a UT-specific BSP. One of those things it shifts is how the specific pass/fail messages are formatted, so the BSP is now in complete control of how these look and what they contain, whether they go to a terminal or log file, etc.
So when the automatic integration build is done it can link against a UT BSP that outputs everything to a log file that is nicely formatted to feed into the report generator.
from osal.
Trac comment by jphickey on 2015-02-25 21:49:57:
Pushed branch "trac-17-osal_basic_ut_framework" containing basic unit test API in OSAL.
Commit [changeset:794cb07]
This modification adds a "ut_framework" library to the OSAL build which implements the core functions of the CFE unit test framework such that it can be used to test any application. It also aims to be a little cleaner and easier to work with, with most stub functions being easily implemented by a macro.
Note 1 - this branch is currently based off my previous ic-merge-jph-all branch. This can be cherry-picked and moved to the development branch when the current IC branch is finalized. It has been pushed here for now so it can be seen/reviewed.
Note 2 - The basic functions added here have been tested individually, but none of the existing unit tests have been modified (yet) to actually use this. Once a good unit test baseline is available, unit tests can then be modified/cleaned up and this would be a good time to transition.
from osal.
Trac comment by glimes on 2015-03-02 18:28:16:
I found myself looking for usage examples. Got any handy?
from osal.
Trac comment by jphickey on 2015-03-03 13:55:43:
As an example, I pushed branch "trac-17-port_osal_core_test", commit [changeset:a9ab7bb]
This shows an example of how the "osal-core-test" would be implemented using the basic framework.
FYI- the original commit (trac-17-osal_basic_ut_framework) was amended to change prototypes and add some ease-of-use macros.
from osal.
Trac comment by jphickey on 2015-03-18 23:08:08:
This is ready for review and integration.
The complete work under this trac ticket is implemented as three separate commits:
[changeset:1e1aa1ac] - This implements the basic UT framework
[changeset:c01e08e4] - This ports the OSAL "core" test to use the framework. Only one test is ported at this point, pending approval from the CCB. The objective here is to prove that the UT framework implemented above is functioning correctly. If agreement is reached that this is the right thing then the rest of the tests can be ported.
[changeset:7cdbe6e0] - This implements stub functions of all the OSAL functions. This is intended to replace the OSAL stub functions implemented in CFE.
from osal.
Trac comment by jphickey on 2015-04-14 11:23:09:
Additional commit added on the trac-17 branch to address an issue with unit tests. When running unit tests with the same pc-linux BSP that normal application code uses, signals are blocked during OS_Application_Startup(). This will prevent timers from operating since these rely on signals.
Typically, a task would be created such that OS_Application_Startup() can return and then signals would be unblocked. Not all of the test code was doing this. This commit fixes those tests that were broken.
Commit: [changeset:ad447ab]
from osal.
Trac comment by jphickey on 2015-06-26 22:40:03:
== REBASED COMMIT IDS ==
The work on this ticket has been rebased to use the UT assert library, which was added under #80.
Disregard all prior commit ids listed
Here is a list of current commits on the branch {{{track-17-rebased-ut-framework}}}
-
[changeset:eefba4bf] Integrates UT assert with the OSAL "BSP" code. Note that this uses //only// the BSP, not the rest of OSAL, for unit test. The BSP provides start up functions (i.e. the "main" abstraction) and UT-specific text/console output routines. This eliminates most {{{#ifdef}}} statements within the test code for support of certain platforms as well as compile time flags for verbosity level as it is all handled by the UT BSP.
-
[changeset:31defa0a] ports all of the existing OSAL "black-box" style tests (the ones under {{{src/tests}}}) to use the UT assert framework. Error counting and test case reporting is now done via a {{{UtAssert}}} call in all of these tests.
-
[changeset:177d954] introduces a new "stub helper" extensions to UT assert that consolidate many of the repetitive operations found in stub functions. This includes a generic way of setting a "deferred" return code (i.e. return X after Y calls) or a forced return code, passing a data buffer (i.e. for read/write stubs), or similar operations. It does this //without// requiring the instantiation of a separate {{{UT_SetRtn_t}}} object so the stub code becomes much cleaner and more manageable.
-
[changeset:23670ac] Adds the generic OSAL stub functions (which are used to test //other// code like CFE) to the OSAL repository under {{{src/ut-stubs}}} directory. These stubs use the helper routines added in the previous commit. This also serves as an example of how those stub helper functions can make the implementations much simpler and more straightforward compared to the prior implementation within CFE ({{{ut_osapi_stubs.c}}}).
Testing Note
In addition to the typical Linux testing, this set (combined 57 + 17 UT assert-based OSAL black-box tests) has also been successfully executed on RTEMS. This is possible due to the BSP integration and the fact that the same support code that can boot the real application on RTEMS can be re-purposed to boot the unit tests as well.
from osal.
Trac comment by glimes on 2015-06-30 11:41:35:
Recommend accept.
I kicked Bamboo and now it's building ic-2015-06-30
which is
currently pointing at the end of this sequence of commits.
Please excuse some very rough patches in the build, I'm in the
middle of shifting between test runner methods.
[edit: finished reviewing tickets. will be working on getting
this cleaned up between now and CCB meeting]
This build is a substantial improvement over development
and
provides the baseline for further improvements.
from osal.
Trac comment by acudmore on 2015-06-30 12:32:31:
Recommend accept. Good idea to keep the OSAL stubs in the OSAL rather than the cFE.
from osal.
Trac comment by sstrege on 2015-06-30 12:40:40:
The ut_osfileapi_stubs.c file also needs to be added to /src/ut_stubs
With that addition I recommend/accept
from osal.
Trac comment by glimes on 2015-06-30 14:29:21:
Per CCB meeting 2015-06-30, accepted for inclusion in development,
after a few small updates we discussed in the meeting.
from osal.
Trac comment by jphickey on 2015-06-30 15:14:23:
Per CCB discussion - I will add the "osfileapi" stubs to this with the caveat that there is no application currently in the build that actually uses these stubs, so the operation of the fileapi stub functions cannot be confirmed at this time.
As a descendant of the #56 fix, this will also need to be rebased after making the updates to that ticket (so the commit ids will change again).
from osal.
Trac comment by jphickey on 2015-07-01 23:27:22:
This has now been rebased again onto a merge that contains the updates for #56 and also incorporates the items discussed at the 2015-06-30 CCB meeting.
The current and hopefully final list of commits on the trac-17 branch:
- [changeset:10b7816]
- [changeset:d6c581c]
- [changeset:9533915]
- [changeset:8be943b] (this also adds the osfileapi stubs)
from osal.
Related Issues (20)
- Add default build/install scripts for regular and test builds.
- Stack size on POSIX needs to account for internal TCB+TLS
- RTEMS `OS_BinSemTimedWait_Impl` and `OS_CountSemTimedWait_Impl` return incorrect status HOT 3
- OS_FileSysAddFixedMap should use the virtual vs physical path name
- OS_GetErrorName Seg Faults HOT 2
- SemGetInfo partial success cases
- Reinstate RTEMS 4.11 build
- count-sem-test, bin-sem-test intermittently fail on POSIX as non-root user HOT 3
- Add generic RTEMS tar file system support HOT 2
- POSIX implementation does not adhere to user-specified stack pointer
- Softsleep is no longer used
- Update OS_TimedRead and OS_TimedWrite to support sub-millisecond timeout resolution
- Static analysis issues JSC 2.1
- Shared workflow scripts need updates for node.js version 20
- Many compilation warnings when compiling under lp64 data model with -Wconversion option HOT 1
- Permit stack allocation from alternative memory pool on VxWorks
- f-sanitizer detect runtime error in coverage-vxworks-console-testrunner
- f-sanitizer detect runtime error in coverage-shared-idma
- Incorrect function parameter types in os-impl-no-select.c HOT 2
- Unit test for scheduler app requires increase in UT_MAX_FUNC_STUBS value HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osal.