jamcar23 / openpilot Goto Github PK

For 0.8.2 I got rid of all the custom UI I had. I'm going to bring it back in parts as I believe 0.8.3 will change the UI again and I don't want to spend a lot of time on it since 0.8.3 will likely be out soon. Anyway...

The follow UI enhancements still need to be done for FP 0.8.2:

• Dev UI from FP 0.8.1 needs to be brought back. At the same time, we should fix / expand the coloring on the stats.
• Port colored lane lines from shane.
• Update spinner to show FP name & version.

Reenable github workflow to build & test on Mac OS.

Discord user duneight got this error when trying to edit indi multi breakpoint source param. At the moment I'm not sure which type of lists / params are broken.

>> Allowed types: list, str
>> This parameter supports live tuning! Updates should take effect within 2.5 seconds
Current value: ['desired_steer_abs', 'vego'] (type: list)
Enter index to edit (0 to 1), or -i to remove index, or +value to append value:
>> 0
Current value: desired_steer_abs (type: str)
Enter index to edit (0 to 16), or -i to remove index, or +value to append value:
>> vego
--------
Must be an integar within list range!
--------
Current value: desired_steer_abs (type: str)
Enter index to edit (0 to 16), or -i to remove index, or +value to append value:
>>

Flexpilot should switch to using semantic versioning with the following convention:

• Major: OP versions
• Minor: Flexpilot feature releases
• Patch: Flexpilot bug fixes

At the same time FP should replace OP's changelog with its own. It can still include OP changes but only when FP's Major version number changes. The managing of this changelog should happen automatically as part of the CI pipeline, as such we'll need a python script to do the following:

• Add commit messages to changelog file.
• Show op param changes: it'll be more helpful to the user and the added changes can come the pr diff of the file.
• Show OP changes when op version updates
• Manage version number based on OP version / by looking at the committed changes and seeing if the messages starts with feat or fix.
• Automatically manage merge conflicts so that changelog is in order based on when the PR got merged

Other things to consider doing at the same time:

• Setup pipeline step to create tags for src & r2++ before merging in a major change.
• Automatically publish github release when r2++ is updated.

Things to think about more:

• What will the version number be for -ci branches? How will we avoid conflicts?

When OP Params are used to enable / disable certain features then it creates a lot of behavior branches in code. Right now, all the CI tests use the default OP Params which means there's a lot of things that aren't being tested as part of the CI pipeline.

This enhancement to rework the CI tests should do the following:

• Each unit and integration test case should run multiple times with each run enabling a different toggleable OP Param.
• The default params (with everything off) should be able to pass comma's integration test for that version of OP.

Things to keep in mind:

• Look into auto-generating tests cases. Ideally we wouldn't have to update a list of toggleable params manually.
• Process Replay (integration tests) need to work with remote comma data and for local data.

Once the poor GPS alert comes up it never goes away, even if GPS has a good signal again.

The purpose of this issue is to outline the ideas, goals, and challenges of rewriting opparams to be its own dedicated process.

Goals

• Allow opparams to evaluate the params internally
  • I shouldn't need to check if the param is enabled everywhere I want to use it, opparams should handle this
  • Ideally opparams doesn't just tell me if it's enabled but can actually return the correct value when it's not
    • e.g. I should be able to replace CP.steerActuatorDelay with op_params.eval(STEER_ACTUATOR_DELAY) (generalized to any prop from any message / state) in any location with no other code changes (import statement for OpParams will still be needed of course)
• Live params in C/C++
  • Selfdrive only, Panda will have to continue to use static / compile time params
• Safer param file access
  • Only 1 process (opparamsd) should read / write the param file(s)
  • Each param should be stored in separate files in order to minimize risk & scope of file corruption

Challenges

• Cereal will have to be used for inter-process communication
  • Will all possible params in opparams be serializable as capt 'n' proto messages?
    • How do we handle params which can't? Just not allow them via opEdit?
    • Will this require different messages for different supported datatypes? If so, how should we handle the different message types in C/C++? (Assuming this is a situation where two different params could be used, i.e. number v. list of numbers.)
  • OpEdit will need to use pub/sub masters to talk to OpParams
    • Will enough of the Param message be serializable so OpEdit can still function (with at least the current level of functionality)?
      • If not:
        • should we use strings and runtime code generation to allow OpEdit to have the full param class?
        • should we have opparamsd handle validation and communicate with OpEdit in a request/response pattern?
• How will opparams know which params are needed to evaluate the param?
  • e.g. to evaluate STEER_ACUATOR_DELAY it needs to look at ENABLE_LAT_PARAMS, ENABLE_STEER_ACTUATOR_DELAY, ENABLE_ACTUATOR_DELAY_BPS_MULTI, STEER_ACTUATOR_DELAY_BP_MULTI, STEER_ACTUATOR_DELAY_V_MULTI, STEER_DELAY_MULTI_BP_SOURCE, ENABLE_ACTUATOR_DELAY_BPS, STEER_ACTUATOR_DELAY_BP, STEER_ACTUATOR_DELAY_V, and STEER_ACTUATOR_DELAY; how will opparams know all this?
  • Perhaps Params could use a lambda / function to handle evaluation on a per param basis?
  • Maybe there should be specific param class for the different types of param groups?
    • e.g. a MultiBpParam which handles evaluating the params
      • if we do this the param group class should be able to declare all need params to make the feature work (e.g. breakpoints will always need BP and V params.)
• Similarly, how will it know which OP message / state to use when evaluating an opparam?
  • e.g. how does it know to use CP.steerActuatorDelay if ENABLE_STEER_ACTUATOR_DELAY is false?
  • Perhaps there is a fallback prop for the param group with a string in the format of [message].[field] which then gets converted to the submaster equivalent string and then uses the same runtime code generation as opEdit to turn it into python?
    • e.g. carParams.steerActuatorDelay string becomes sm["carParams"].steerActuatorDelay string where sm is an existing SubMaster instance inside of opparamsd once the code is generated
• Automated tests will be harder as all processes would now depend on opparamsd (and whatever it depends on)
  • how will this affect #78?

Implementation Ideas

• The existing opParams class & functionality will be need to be split across at least 2 classes: OpParamsd and OpParamsListener (final names tbd)
  • OpParamsd
    • contains all functionality related to reading / writing param files, evaluating params, and anything else related to the management / control of the params directly
    • should publish 3 messages: OpParamsEvaluation, OpParamsRaw, OpEditResponse (final names tbd)
      • Evaluation should contain the pre-evaluated params
        • used by processes (controlsd, planner, etc.) which only need to know the final value that should be used
      • Raw should contain the complete Param class
        • used by processes (OpEdit, UI (eventually)) which need to know / display all param information
      • OpEditResponse will be the response from opparamsd when OpEdit request some param change
        • not 100% sure what will go here or what will be needed
        • if OpEdit has its own complete copy of the Param it may be able validate the new values inside its own process in which case a response may not be needed
    • should subscribe to all OP messages + any new messages needed for OpEdit (e.g. OpEditParamChangeRequest)
  • OpParamsListener
    • this will be the version of OpParams that is used outside of opparamsd
    • publishes 0 messages; subscribes to OpParamsEvaluation (perhaps OpParamsRaw optionally too)
    • should contain all code needed for single line replacement of any OP prop in any location (goal 1.2)
• OpEdit will need to be reworked to keep existing functionality while not using OpParams directly
  • Subscribes to OpParamsRaw and OpEditResponse (if needed)
  • Publishes OpEditParamChangeRequest
    • used by opparamsd to listen for when the user is changing param values
  • In addition to things like check if the entered value has the correct expected type, OpEdit will need to validate that the data type and structure can be serialized as a capt 'n' proto message. These can be done two ways:
    • 1st. OpEdit can send a change request to opparamsd which then validates the change request and then sends a response back to OpEdit
      • this would absolutely require the request & response messages as OpEdit has to alert the user in the event of errors
      • this would probably be the more performant option (at runtime)
    • 2nd. OpEdit could use the data inside OpParamRaw to generate its own copy of the Param python classes and then validate new values directly
      • this would essentially use the same Param code as opparamsd without any of the read / write functionality (or the ability to update the params used by other processes)
        • since OpEdit would be creating a second copy of the params classes (as well as possibly using runtime code generation) option 1 is probably more performant
      • a response message may not be needed because OpEdit already knows the data is valid and will be accepted by opparamsd

Reenable the github workflow for build & testing webcam.

Create a spinner that shows the current version number & fork name. This will QT so it needs to be after 0.8.3.