Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Organizational review #236

Open
7 of 16 tasks
Funth0mas opened this issue Nov 2, 2018 · 22 comments
Open
7 of 16 tasks

[WIP] Organizational review #236

Funth0mas opened this issue Nov 2, 2018 · 22 comments
Milestone

Comments

@Funth0mas
Copy link
Contributor

Funth0mas commented Nov 2, 2018

Organizational review summary:

  • Add canmatrix.formats package
  • Unify/define formats interface, consider to use zope.interface or simply abc.abstractclass
  • canmatrix.canmatrix module should be renamed - canmatrix.core ??
  • Possibly expose a particular few things via canmatrix.some_thing() - what classes ??
  • Decide and apply a consistent import 'format'
    • No from module import * (!)
  • Isolate CLI from library rather than colocating (use cli or cmd module?)
  • Use CLICK instead of optparse
  • pathlib2 or os.path ?
    • Well, decide one way or the other, altendky preference is for pathlib stuff.
  • snake_case for all except of classes, classes with UpperCamel - in progress
  • loadb for loading any format from a byte sequence
  • static typing
  • mypy running in CI
  • check and resolve class-wise duplicities (del_frame() vs remove_frame(), comment vs comments)
  • remove useless cm prefix from some files
  • ??
@altendky
Copy link
Collaborator

altendky commented Nov 3, 2018

I'm not big on the cm prefixes as we already have the canmatrix package (the outermost one). It would be nice to never canmatrix.canmatrix as the second one really describes nothing useful.

As to interfaces, my impression from discussions in #python is that zope.interface is the thing to learn from and use, not stdlib (such as abc).

Anyways, you are talking about moving canmatrix.dbc to canmatrix.converters.dbc? If so they are presently referred to as formats I think?

@altendky
Copy link
Collaborator

altendky commented Nov 3, 2018

If we're talking about an overall organizational review (I'd rather make an overall plan first then shift everything around, I'm sure some opinions will change as we discuss various points) then I'll add that we might need a misc/utils or whatever module for things like grouper() (unless we use one of the libraries that provides tools like this).

https://github.com/ebroecker/canmatrix/pull/224/files#diff-c1624d42e3750ac47cb2bbd3c178a5d0R536

@Funth0mas
Copy link
Contributor Author

Bad name, of course I thought formats.
Let cancel this ticket and make it somehow better...later... But I'm afraid I won't help a lot with it :-(

@Funth0mas
Copy link
Contributor Author

Concerning the prefix, only now I get the cm stands for 'canmatrix', not 'converter'. Yes, I'm not analphabet, I see the converter is not comverter :-) . Does not matter.

@altendky
Copy link
Collaborator

altendky commented Nov 3, 2018

We can rename the ticket or make a new one, no matter to me. I see a few things at the moment, let's build the list if there are more. If we are expecting to get more tests in place soone than later, maybe we continue that first before any reorganization.

  • Add canmatrix.formats package
  • canmatrix.canmatrix module should be renamed
  • Possibly expose a particular few things via canmatrix.some_thing()
  • Decide and apply a consistent import 'format'
    • No * imports
  • Isolate CLI from library rather than colocating
  • pathlib2 over os.path
    • Well, decide one way or the other, my preference is for pathlib stuff.
  • lowerCamel or snake_case?

@ebroecker
Copy link
Owner

my 2 cents about overall organizational review

  • I want to create a release (0.7) more or less the current development state (gonna check if some bugs should be fixed)
  • after that I am open to abandon python2 support
  • after that I am very open for cleaning up the code and reorganizing it
  • certainly it would also be an option to work for these things in a branch.

@altendky , @Funth0mas : What would you prefer?

@altendky
Copy link
Collaborator

altendky commented Nov 5, 2018

  • This isn't urgent for me so waiting until after a release seems fine.
  • I expected to give more notice than that for dropping py2. I don't know how we go about making sure people notice but I would say we at least have a release which notifies of imminent loss of py2 support before releasing a version that doesn't support py2 (or makes no promises of support for it). I only brought it up because I don't want to be continuing to support it in 2025 simply because we didn't get around to warning anyone it was going away.
  • Definitely do it in a branch. Definitely don't let the branch linger. Yeah, git can handle moves but it's just another layer and let's not go begging for merge trouble.

How about we add another thing before a reorg. I don't recall offhand how inconsistent we are but let's pick an import style policy and implement it first. I personally nearly always use the form import canmatrix.formats then canmatrix.formats.load(). As opposed to from canmatrix.formats import load or import canmatrix.formats as f or from . import formats or... If we are reorganizing we'll be changing imports. If we are changing imports we might end up changing lots of names used in code. Mixing it all together seems not great. (I added this to the list of overhauls to consider)

@Funth0mas Funth0mas changed the title Move converters to own module? [WIP] Organizational review Nov 7, 2018
ebroecker added a commit that referenced this issue Nov 7, 2018
import issues will generally reworked in #236
@altendky
Copy link
Collaborator

added the following to the list

  • Isolate CLI from library rather than colocating

@Funth0mas
Copy link
Contributor Author

  • don't use import *

This was referenced Nov 13, 2018
@ebroecker
Copy link
Owner

@altendky
Copy link
Collaborator

@ebroecker, I saw this as building a list of 'policies'. Do we need an 0.8 ticket? Or milestone? (never worked with milestones in github) or tag?

@ebroecker
Copy link
Owner

I thought it's right here because the policy would be to use "byte sequences". If not - just ignore...

@ebroecker ebroecker added this to the 0.8 milestone Feb 16, 2019
ebroecker added a commit that referenced this issue Feb 22, 2019
#236
* refactor api to snail_case
* rename BoardUnits to ecus
* fix some utils
ebroecker added a commit that referenced this issue Feb 22, 2019
fixes #230 
parts from #236
* add support for long ECU names
* fix long enviroment names
refactor: remove import *
refactor: create small sub-functions for dbc creation
ebroecker added a commit that referenced this issue Feb 23, 2019
* refactor/rename
* adapt examples to new api
* remove  import * and refactor modules
#236
ebroecker added a commit that referenced this issue Feb 26, 2019
@Funth0mas
Copy link
Contributor Author

@altendky the 'isolate CLI from library' means to move the two CLI scripts to another package, kind of canmatrix.cli, and update the `setup.py?

@Funth0mas
Copy link
Contributor Author

I investigated the zope.interface and as far as I can see it has no real benefit unless we would like to use zope component system, what might be an overhead. Moreover the zope.interface support for mypy static typecheck is still in progress, actually leading to lot of mypy errors.
The zope interface is not meant to enforce an interface or even verify that this interface is implemented. I think the abc.abstractclass is pretty enough to define, that there is (can be) load and dump function in each format.
What I find more important is to refactor the formats to become classes, not only modules. What kind of interface definition to use is than minor question.

Funth0mas added a commit to Funth0mas/canmatrix that referenced this issue Sep 17, 2019
Funth0mas added a commit to Funth0mas/canmatrix that referenced this issue Sep 18, 2019
ebroecker pushed a commit that referenced this issue Sep 18, 2019
Funth0mas added a commit to Funth0mas/canmatrix that referenced this issue Sep 18, 2019
Funth0mas added a commit to Funth0mas/canmatrix that referenced this issue Sep 27, 2019
@altendky
Copy link
Collaborator

I know I'm way way late here...

I investigated the zope.interface and as far as I can see it has no real benefit unless we would like to use zope component system, what might be an overhead. Moreover the zope.interface support for mypy static typecheck is still in progress, actually leading to lot of mypy errors.

Certainly zope.interface is usable on it's own. It lets you describe an interface and express that something else satisfies that interface. All without being super infectious and annoying.

The zope interface is not meant to enforce an interface or even verify that this interface is implemented. I think the abc.abstractclass is pretty enough to define, that there is (can be) load and dump function in each format.

You can write tests which verify implementation of an interface. On the other hand yes you can use ABC to infect everything with a metaclass which basically keeps anyone else that uses this from using their own metaclasses. This exclusivity alone is sufficient to make me walk away from metaclasses. Though yes, ABC can be used without the metaclass at which point it's a lot more similar to zope.interface in that you do separate explicit checks that a given class implements an interface. Though I still don't end up seeing ABC recommended much.

Anyways, as much as I haven't been actively following canmatrix this year I have been watching enough to see all the work you are putting into it. Thanks for helping keep it improving, @Funth0mas .

@Funth0mas
Copy link
Contributor Author

Thanks for the comment, maybe we need neither zope nor abc, just plain classes. Still better than functions on module level sharing data over global variables and fighting with python import system.

Next refactoring steps could be renaming canmatrix,py to core.py, finish codestyle cleanup on this file (snake_case etc.; thanks to mypy coverage it's quite secure job now) and than maybe first moving formats to classes. But such change will make the API completely incompatible.

@ebroecker
Copy link
Owner

@Funth0mas
I agree.
But I want to do a release just before making the API incompatible.

What do you think about releasing more or less "as is" as canmatrix 1.x
And start with new API and without py2 release this laster as canmatrix 2.x?

What do you think about:

  • create a new release from current state as canmatrix 1.0
  • afterwards we use development branch, kick out py2 and start with your proposed tasks

@altendky
Copy link
Collaborator

So a major version number that will only ever (be planned to) have a single release? Seems a bit funny, but maybe it doesn't matter. This seems like maybe just more pre-1.0 thrash. But, I really don't feel strongly about this. To a degree I don't really care about version numbers. I've switched to calver just to avoid thinking about this (yeah, I should have a deprecation policy but... my code just isn't that tidy yet).

@altendky
Copy link
Collaborator

Oh, as to using some interface tool or not, I see it as an optional extra like type hints. It isn't needed for things to work, it can provide some more clarity to both library and application developers.

@ebroecker
Copy link
Owner

... the interface for the formats is quite simple:

  • load()
  • dump()

using zope for just defining load and dump seems a bit overengineered to me.
Maybe I'm wrong. What do you have in mind while using something like zope?
How could it look like, perhaps I just did not understand the concept yet.

About versions:
My idea was like having a 1.x branch which is py2 compatible and a 2.x+ branch which is only py3.
I'm also OK with merging these things above in pre-1.0, but if so, I think, we should support py2...
I'm also OK with switching to calver or whatever, maybe if we get automatic releases work (related to #291)

@altendky
Copy link
Collaborator

altendky commented Oct 6, 2019

Classes were what was in my head. I didn't bother to think through it recently though. But, the act of even trying to define the interface sometimes is useful. Do we really want all this stuff on one class? Anyways, my recent response wasn't to say I thought defining interfaces formally should be a priority.

@Funth0mas
Copy link
Contributor Author

Just my thoughts... Convert formats from module level functions to the classes would be useful for getting rid of the global keyword in yaml, xlsx, sym and arxml formats and also of the lazy _init_yaml in the yaml format. I believe that the py2 support is no pain for such refactoring. Pain is to well define it...
Similar for the code style cleanup, it should not break py2.
Maybe I'm wrong but even if you drop py2, you can't use new features coming in py 3.6, 3.7, so currently dropping py2 shouldn't be that high prio. (I understand "drop py2" as "find and remove all workarounds as future library or sys.version conditions)

BTW, @ebroecker , the task "loadb for loading any format from a byte sequence" above is still valid, or already resolved? Thanks.

ebroecker pushed a commit that referenced this issue Oct 14, 2019
ebroecker added a commit that referenced this issue Nov 8, 2019
* include LICENSE in source distribution (#365)
* [arxml]: rx/tx typo fix 
* annotate and fix codestyle dbc (#369)
* Always use Signal.is_little_endian as bool #326 (#371)
* Add coding, remove #!python where inappropriate (#370)
* annotation cleanup ticket #323 (#372)
* Update cli doc (inspired by #361) (#373)
* fix: xls is using the wrong number for arbitration id (#377)
* add outputformat for Scapy (#378)
* fix special char handling in scapy exporter
* Dump factor, offset, min, max as float. Load as numeric (#380)
* added --jsonNativeTypes as cli option
* [arxml]: merge container PDU and Secured PDU support  (#384)
* [arxml]: support for containers
* [arxml]: create signalgroups for containers
* [arxml]: support secured pdus (#383)
* fix for wrong default ArbitrationId handling (#386)
* fix for #381 wrong default ArbitrationId handling
* optional ignoring failures durig character encoding #375 (#388)
* fix for #238 (j1939 Frame setters might raise TypeError) (#389)
* move j1939 (pgn, prio, source) handling direct to arbitration-id class (fixes #238)
* [arxml]: #385 // Arxml Container support and ARXML refactoring (#390)
* [json] fix multiplex reading issue (#136)
* [arxml]: refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now fits to next correct dlc value, DBC export reniced for candb++ (#385)
* [arxml]: refactoring and better type handling #368
* [dbc]: fix #242. J1939 and FD Frames in dbc
* [xlsx]: fixes #240 (signal init value in excel template)
* [xlsx] feature for for #288 (#393) separate lines for values
* switch from optparse to click (#236) (#394)
* add Basic cli tests (#395)
* Annotation cleanup #323 (#396)
* Remove cm prefix (#236) (#399)
* add muxing support to scapy output; merge PR #398 also (#400)
* [try to] Unify imports (#402) unify imports (#236)
* [wireshark] dump/export wireshark lua can subdissector (#404)
* add cannelloni dissector for wireshark
* cycle_time as generic signal and frame attribute  (#405)
* implement cycle_time as generic signal and frame attribute / remove Gen[Msg/Sig]cycle from attributes  #146
* use effective frame cycle time for export use gcd of singnal-cycletimes for calculating effective cycle time
* [dbc] Iss407: Signal Comments with space bevore semicolon broken import (#409)
* [dbc] fix for issue #407 Signal Comments with space bevore semicolon broken import
* make initial value to a native attribute of Signal class (#408)
* [dbf] exended ids corrected
* [dbc] optional allow multiple singals with same name in frame #411 (#412)
* Fix setup.py for console_scripts (#417)
* [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416)
* fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if
native attribute exists.
* convert: selective rx/tx ecu extraction issue #421 (#422)
* [arxml] can_fd info (potentional fix for #410) (#418)
* add baudrate attributes
* Auto deploy (#420)
* add auto deploy for tags on master
ebroecker added a commit that referenced this issue Sep 29, 2020
* include LICENSE in source distribution (#365)
* arxml: rx/tx typo fix 
* arxml ipdu-groups fix
* annotate and fix codestyle dbc (#369)
* Always use Signal.is_little_endian as bool #326 (#371)
* Add coding, remove #!python where inappropriate (#370)
* annotation cleanup ticket #323 (#372)
* Update cli doc (inspired by #361) (#373)
* fix: xls is using the wrong number for arbitration id (#377)
* add outputformat for Scapy (#378)
* simplify get_fmt
* Dump factor, offset, min, max as float. Load as numeric (#380)
* added --jsonNativeTypes as cli option
* ARXML: merge container PDU and Secured PDU support   #381
* #382 support secured pdus
* Iss381 - fix for #381 - wrong default ArbitrationId handling (#386)
* optional ignoring failures durig character encoding #375 (#388)
* fix for #238 (j1939 Frame setters might raise TypeError) (#389)
* move j1939 (pgn, prio, source) handling direct to arbitration-id class
(fixes #238)
* Iss385 // Arxml Container support and ARXML refactoring (#390)
* ARXML refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now
* fits to next correct dlc value, DBC export reniced for candb++ (#385)
* DBC: fix #242. J1939 and FD Frames in dbc
* XLSX fixes  (#392)
* fix for #288 
* switch from optparse to click (#236) (#394)
* Remove cm prefix (#236) (#399)
* add muxing support to scapy output; merge PR #398 also (#400)
* add basic wireshark dissector creator
* add cannelloni dissector for wireshark
* implement cycle_time as generic signal and frame attribute / remove
Gen[Msg/Sig]cycle from attributes  #146
* use effective frame cycle time for export
* fix for issue #407 [dbc] Signal Comments with space bevore semicolon broken import
* make initial value to a native attribute of Signal class (#408)
* [dbf] exended ids corrected
* optionally allow multiple singals with same name in frame #411 (#412)
* Fix setup.py for console_scripts (#417)
* [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416)
* fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if
native attribute exists.
* [WIP] [ARXML] can_fd info (potentional fix for #410) (#418)
* [ARXML] can-fd recogintion
* add baudrate attributes
* [convert] add delete_obsolete_ecus (fix for #434) (#438)
* [copy] Iss430 better attribute handling  (#437)
* handling missing default values for defines
* add ecu attribute handling #430
* fix for #441 jsonAll export option working again
* make old test less loud
* Fix some bugs in FIBEX export (#445)
* Arxml limits number formats (#457)
* Check for EOF when parsing enums in a .sym file. Fixes #465 (#468)
* Fix test to check for different exception in the python2.7 case
* Log the exception as well when importing a format failed (#467)
* Implement processing of the Title property of .sym files.
* Fix CanMatrix.attribute to return default as documented
* Set the default for the "Title" attribute properly
* Reimplement the quote_aware_comma_split function to handle spaces
between fields
* Change the sym parser to use quote_aware_comma_split to split enum value
tables. Fixes #462.
Add test cases to show that enums are read in correctly from .sym files
* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474 (#475)
* Add enumberations as a field to the json file to store the value tables (#476)
* Adding changes to allow ArbitrationId classes to be sorted (#466)
* Adding comparators to ArbitrationId
* Changing default Arbitration ID extended to False
* Allow parsing of the main .sym signal types : string, raw, char.
* Allow parsing of the main .sym signal types : string, raw, char.
* Add fix for "isidentifier" fucntion on python 2.7
* Fix logic to match variable name. (Functionality unchanged)
* Accept types which are in the value table as enum sgnals (#482)
* Fix additional case in sym enum export (#486)
* Accept types which are in the value table as enum sgnals
* Fix previous change to also generate enums from the signal values field
* [DBF] support j1939 read (#501)
* [arxml] add some basic support for flexray and ethernet data
* prove of concept for flexray dump (#432)
* Read Baudrate from arxml
* Enable reading hex values with prefix 0x
* #496 fix ignore encoding switch
* decode_number now supports floating point value
* dbf fix for exporting empty matrix
* clean encoding error implementation

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
ebroecker added a commit that referenced this issue Jan 11, 2022
* include LICENSE in source distribution (#365)

* arxml: rx/tx typo fix 

fix for #366

* annotate and fix codestyle dbc (#369)

* Always use Signal.is_little_endian as bool #326 (#371)

* Add coding, remove #!python where inappropriate (#370)

* annotation cleanup ticket #323 (#372)

* Update cli doc (inspired by #361) (#373)

* fix: xls is using the wrong number for arbitration id (#377)

* add outputformat for Scapy (#378)

* add output format for scapy
* fix special char handling in scapy exporter
* add unit test
simplify get_fmt

* Dump factor, offset, min, max as float. Load as numeric (#380)

* Dump factor, offset, min, max as float. Load as numeric

* fixed tests

* fixed tests

* added --jsonNativeTypes as cli option

* review remarks

* ARXML: merge container PDU and Secured PDU support  (#384)

* #381 support for containers
* #381 create signalgroups for containers
* #382 support secured pdus) (#383)
* add unittests for #363 and #382

* Iss381 - fix for #381 - wrong default ArbitrationId handling (#386)

* fix for #381 wrong default ArbitrationId handling

* support for Scapy input file (#387)

* add output format for scapy

* optional ignoring failures durig character encoding #375 (#388)

* fix for #238 (j1939 Frame setters might raise TypeError) (#389)

move j1939 (pgn, prio, source) handling direct to arbitration-id class
(fixes #238)

* Iss385 // Arxml Container support and ARXML refactoring (#390)

ARXML updates:
* fix for #385 
* json fix multiplex reading issue (#136)
* ARXML refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now
fits to next correct dlc value, DBC export reniced for candb++ (#385)
* add test for fit_dlc
* arxml refactoring and better type handling #368
* improvement for #242

* DBC: fix #242. J1939 and FD Frames in dbc

candb seems to be queasy about the order of the VFrameFormat attribute
definition. This Patch seems to generate a dbc, which can contain can_fd
AND J1939-frames.

* XLSX fixes  (#392)

XLSX: fixes #240 (signal init value in excel template)
XLSX: fixes #367 (2.)

* fix for #288 (#393)

* switch from optparse to click (#236) (#394)

* [WIP] add Basic cli tests (#395)

* add tests for global, manipulation and filter swithces of cli.convert
* add basic cli test for cli.compare

* fix some bugs, which were found by these tests

* Annotation cleanup #323 (#396)

* Remove cm prefix (#236) (#399)

Thanks

* add muxing support to scapy output; merge PR #398 also (#400)

* [try to] Unify imports (#402)

* unify imports (#236)

* [WIP] dump/export wireshark lua can subdissector (#404)

* add basic wireshark dissector creator
creastes lua script for dissecting can messages

* add cannelloni dissector for wireshark

* add some doc for scapy and wireshark usage

* [WIP] cycle_time as generic signal and frame attribute  (#405)

* implement cycle_time as generic signal and frame attribute / remove
Gen[Msg/Sig]cycle from attributes  #146
* use effective frame cycle time for export
use gcd of singnal-cycletimes for calculating effective cycle time

This removes one (of many) dbc specific parts from canmatrix and other formats

* Iss407: [dbc] Signal Comments with space bevore semicolon broken import (#409)

* fix for issue #407 [dbc] Signal Comments with space bevore semicolon broken import

* make initial value to a native attribute of Signal class (#408)

* make initial value to a native attribute of Signal class

* [dbf] exended ids corrected

* optionally allow multiple singals with same name in frame #411 (#412)

* Fix setup.py for console_scripts (#417)

* [WIP] [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416)

* optionally allow multiple singals with same name in frame #411
* fixes #413 (space before semicolon mulit line comment)
* fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if
native attribute exists.

* convert: selective rx/tx ecu extraction issue #421 (#422)

* [WIP] [ARXML] can_fd info (potentional fix for #410) (#418)

* potentional fix for #410
* [ARXML] can-fd recogintion
* add baudrate attributes

* [WIP] Auto deploy (#420)

* add auto deploy for tags on master

* add submodules doc

* update doc to make start_bit more clear in documentation #424

* [dbc] fix dbc issues (#431)

* fix dbc export for empty matrix
* copy_frame handles default values now (fix #430)

* fix copy_frame

* fix for issue #436 (#439)

* [convert] add delete_obsolete_ecus (fix for #434) (#438)

* [copy] Iss430 better attribute handling  (#437)

* handling missing default values for defines
* add ecu attribute handling #430

* fix for #441 (#442)

* fix for #441 jsonAll export option working again

* make old test less loud

* Fix some bugs in FIBEX export (#445)

* Fix FIBEX convertion, Handle duplicate signal name

* Repair more click convert parameters (#451)

* Repair command line param names for csv, json, xls(x)

* update tests

* use last supported colorama for pypy3.4

* Fixing issue where we see dictionary attributes change during iteration (#455)

* Arxml limits number formats (#457)

* add number decoding in arxml for different basis
* fix #454

* fix for #460

* fix #460

* Update cli.rst

add arxmlExportVersion documentation  (#461)

* Check for EOF when parsing enums in a .sym file. Fixes #465 (#468)

* Check for EOF when parsing enums in a .sym file. Fixes #465

* Fix test to check for different exception in the python2.7 case

* Log the exception as well when importing a format failed (#467)

* Process the .sym file "Title" field. fixes #469 And Fix Enum dictionary comma splitting  (#470)

* Implement processing of the Title property of .sym files.
* Fix CanMatrix.attribute to return default as documented
* Set the default for the "Title" attribute properly
* Reimplement the quote_aware_comma_split function to handle spaces
between fields
* Change the sym parser to use quote_aware_comma_split to split enum value
tables. Fixes #462.
Add test cases to show that enums are read in correctly from .sym files

* fix for #471 (#473)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474 (#475)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474

Remove the criteria that PF must be >= 240 for PS field to be set as
this seems to be incorrect. (It may be that for some specific case the
PS need to not be used, but it isn't documented here and in general the
PGN should be all 18 bits)
Modifed Tests:

test_canmatrix_get_frame_by_pgn()
seemed to be using made up data and didn't accound for the DP and EDP
fields in expected PGN

test_canid_repr()
Modified the expected repr value to include the PS field in the PGN.

test_canid_parse_values()
Modified expected PGN to include the PS field, Moved the list of
test_data into parametrized cases on a new test.

New Tests:

test_frame_j1939_id_from_components()
Tests constructing a full 29-bit CAN ID from the Priority, Source, and
PGN

test_frame_decode_j1939_id()
Tests extracting Priority Source and PGN from an Arbiration ID

* Add the global desistantion Address 0xFF to the BAM arbitration IDs

* Improve the fix, to more explicitly handle the difference between
PDU-Format 1 and PDU-Format 2, (where the PF >= 240 comes in).
The PGN setter now will now set the arbitration ID to contain the
destination address, but when the PGN getter is used it will strip the
destination field off. I think that it is unlikely for the PGN setter
to be used with PDU-F1 PGN containing a destination. But in the case
that it is I think it makes more sense to not throw the destination
address field away.

* Add enumberations as a field to the json file to store the value tables (#476)

Add a test to check that if a json file is read containing enums and
then saved and re-read it get the same enum table

* Adding changes to allow ArbitrationId classes to be sorted (#466)

* Adding comparators to ArbitrationId
* Changing default Arbitration ID extended to False
* Review based changes
Co-authored-by: Syed Raza <[email protected]>

* Parse sym types (#479)

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Add fix for "isidentifier" fucntion on python 2.7

* Fix logic to match variable name. (Functionality unchanged)

Co-authored-by: Eduard Bröcker <[email protected]>

* Fix sym.dump to export the value_table into the {ENUMS} field of the (#481)

.sym file.
Refactor to remove the use of global variables "enum_dict" and "enums"

Add a test which checks that enums in the original .sym file and enums
which appear as values on a signal are exported to the .sym file.

* Accept types which are in the value table as enum sgnals (#482)

* Fix additional case in sym enum export (#486)

* Accept types which are in the value table as enum sgnals

* Fix previous change to also generate enums from the signal values field
when signal is in a multiplexed frame.

* [DBF] support j1939 read (#501)

Co-authored-by: Broecker <[email protected]>

* possible fix for #490 (#503)

* add tests for issue #424 (#500)

add tests for Motorola forward MSB and Motorola forward LSB start_bit

* Iss484 (#502)

* [arxml] add some basic support for flexray and ethernet data

* extract most of relevant flexray info (#432)

not yet integraded - only extracted by now

* prove of concept for flexray dump (#432)

* starting rework ARXML

* add xlsxwriter to test deps

* add pyyaml to test-reqs

* Update requirements.test.py3.txt

* fix so that tests work again

* remove py3.4 add py3.8

* disable py34 test - enable py38 test

* disable py34

* remove py3.4

* remove py8 warnings

* fix dbc for py8

* fix for #484

buggy ARXMLs with no System-Signals referenced

* fix ci

* fix ci (#504)

* fix ci

* Iss492 (#505)

* integrate fix from #492

* Arxml read baudrate (#452)

* Read Baudrate from arxml

* Enable reading hex values with prefix 0x

* Adaption to pass test with string speed values

* Update to read in hex values define with 0x and integers e.g. 0

* Removal of commented line

* Explicit conversion to string for logger output

Co-authored-by: ForestRupicolous <>
Co-authored-by: Eduard Bröcker <[email protected]>

* Iss496 (#506)

* #496 fix ignore encoding switch

* Iss499 (#507)

* fixes #499

decode_number now supports floating point value

* [WIP] [arxml] add some basic support for flexray and ethernet data (#426)

* [arxml] add some basic support for flexray and ethernet data
* extract most of relevant flexray info (#432)
* starting rework ARXML
* fix dbc for py8
* support for container-pdus with "none" type
* fix for container id byteorder
* fix for secured-ipdus in container
* fix for byteorder big endian header_ids of container-i-pdus
* once again header_id
* Header_id again
* fix for missing compu method
* make SOME/IP at least work a bixt #283
* interprete OPAQUE as intel
* dbc allow '"' in Values
* fix container-pdu without header but with offset
* arxml: fix for container-ipdus, containing same pdu multiple times
* fix enum support for py2.7

* Iss509 (#510)  dbf fix for exporting empty matrix

* dbf fix for exporting empty matrix

* Iss496 (#511)

* #496 fix ignore encoding switch
* clean encoding error implementation

* Master (#514)

* remegerge release (#512)

* Fix sym mux decode, by always treating mux field as unsigned (#517)

* Add a test to show that sym file mux values arent working

* Add a fix to always treat MUX values as unsigned signals

* Log as warning instead of exception (#519)

Logging as exception prints a stack backtrace. Since the modules are optional
and listed as extra instead of a required dependency, this should probably be
a warning instead of an error/exception.

Co-authored-by: Lennart Moltrecht <[email protected]>

* src/canmatrix.py: fix nested mux with no values (#527)

If a complexly multiplexed signal has a sub-multiplexer with no values, a TypeError is thrown.

* add is_fd flag for json output (#532)

Co-authored-by: Dennis Röck <[email protected]>

* Iss526 (#536)

* fix ci

* fix ci

* fixes #526

* fix for #526escaped quotes

* #526 use raw strings for test

* Bump lxml from 4.5.2 to 4.6.2 (#539)

Bumps [lxml](https://github.com/lxml/lxml) from 4.5.2 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](lxml/lxml@lxml-4.5.2...lxml-4.6.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Iss535 (#537)

* fix ci
* possible fix for issue #535

* Iss522 (#530)


* arxml: add reciever information from signals to frames  (issue #522)

* Fix ci (#542)

* fix dbc testcase
* remove xlsx read from tests (#541)

* basic ldf import

* update requirements for ldfparser

* first implementation for odx import#544

* start diag frame at byte 1

* odx service muxer

* odx zwischenstand

* issue #531 Lin Converters

add support for LIN-Converters

* odx zwischnstand

* Revert "odx zwischnstand"

This reverts commit f4476de.

* Update xls.py (#549)

Thanks!

* add direct decode function for python can frames (#548)

* better mux export hanling in sym

* Correctly the result if compare two frame with different ID (#552)

When comparing two dbc, the the frame has different ID it should be marked as "changed" frame.

* Overall code improvements (#555)

* fix: use six.moves instead of future.moves to fix annoying import of top level modules

* fix: missing future import + clean up dependencies

* fix: __package__ does not exist in Python 2.7

* fix: string decoding issue in Python 2.7

* update tox.ini

* fix: install all extras in tox.init

* update travis and appveyor config

* remove unused requirements files

* update tox.ini

* remove unwanted line in test.sh

* fix: Python version 3.9 not available in AppVeyor yet

* fix: only keep minimum requirements for test

* improvement dbc to dbf (#556)

temporary hot fix for canconvert dbc to dbf

* fix issue #547 (#560)

DeprecationWarning: The usage of `cmp` is deprecated and will be removed
on or after 2021-06-01.  Please use `eq` and `order` instead.
    @attr.s(cmp=False)

Signed-off-by: An Nguyen <[email protected]>

* fix issue #559 (#561)

Signed-off-by: An Nguyen <[email protected]>

* fix initial value setting (#566)

Thanks
Co-authored-by: JAZI Nadhmi <[email protected]>

* Set initial_value to physical value (#567)

Currently the initial_value is incorrectly set to the signal raw value.

* arxml fix (#564)

Co-authored-by: Eduard Bröcker <[email protected]>

* arxml.py: (#569)

optimize find/findall
    fix ECU transmitter/receivers for clusters
    rework signals receiers settings

Co-authored-by: JAZI Nadhmi <[email protected]>

* [arxml]: Use ISignal Name for canmatrix.Signal (#570)

Add attributes to  Signal for other names

Co-authored-by: JAZI Nadhmi <[email protected]>

* ARXML Replace ArTree with more simple cache

add option to rename Signal from Signal-Attribute

* add/update doc

* Arxml refactoring (#571)

* ARXML Replace ArTree with more simple cache
* add option to rename Signal from Signal-Attribute
* add/update doc

* rework arxml parsing and use Pdu for PDU contained frames (#576)

Add a converter from PDU contained to multiplexed frame
Add option to handle it in the CLI
Update decoder function

Co-authored-by: JAZI Nadhmi <[email protected]>

* fix convert pdu container (#577)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Remove prints from the code and add the message to the exception (#578)

* remove duplicated pdus (#583)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Improve multiplexer JSON output (#585)

Co-authored-by: Edward Pierzchalski <[email protected]>

* Fix typo (#586)

* bugfix: to get the signals for a pdu you have to use follow_all_ref instead of follow_ref (#591)

Co-authored-by: Markus Konrad <[email protected]>

* improve logging output to be able to identify problems faster/better (#592)

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

* fix for issue #596 (#597)

cope with signals without receiver

Co-authored-by: Eduard Bröcker <[email protected]>

* fix for #403

* arxml init value may not decoded

just ignore #550

* fixes #572

* arxml helpers

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix PDU container decoding (#605)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Kankan patch1 (#604)

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix tox and continuous integration (#609)

* Fixing Issue#610 :: Custom Signal Attributes are removed due to wrong indent

* Kankan patch :: Custom Signal Attributes are removed  (#611)


* Fix #610 :: Custom Signal Attributes are removed due to wrong indent

* #612 Bugfix formats.xls and formats.xlsx missing method parameters (#613)

* Update guess_value method in utils.py (#615)

add a conversion of 0b and 0x string values into int string representation in the guess_value method

initial motivation was to get valid input values for the decimal.Decimal() class
since it crashed in the arxml.py file line 1234:

1232            if initvalue is not None and initvalue.text is not None:
1233                initvalue.text = canmatrix.utils.guess_value(initvalue.text)
1234               new_signal.initial_value = float_factory(initvalue.text)

* Added deleteFloatingSignals , checkFloatingFrames, checkSignalRange, checkSignalUnit, checkSignalReceiver & checkFloatingSignals

* Added J1939 to Extended dbc and vice versa

* feat: add support for Python 3.10 (#619)

* feat: add support for Python 3.10
* chore: fix appveyor configuration
* chore: remove useless commands in appveyor configuration
* chore: Python 3.9 and 3.10 only available for Visual Studio 2019 and up in appveyor

* arxml: read data related to E2E-protection (#594)

* read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object
* also read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object if a Container-PDU is processed

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Bröcker <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Broecker <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Lennart Moltrecht <[email protected]>
Co-authored-by: Ryan Rowe <[email protected]>
Co-authored-by: Dennis <[email protected]>
Co-authored-by: Dennis Röck <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cesare12 <[email protected]>
Co-authored-by: Moris Yun <[email protected]>
Co-authored-by: Darkin <[email protected]>
Co-authored-by: jazi007 <[email protected]>
Co-authored-by: pierreluctg <[email protected]>
Co-authored-by: L X <[email protected]>
Co-authored-by: JAZI Nadhmi <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Ajinkya Pasalkar <[email protected]>
Co-authored-by: tainnok <[email protected]>
Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>
Co-authored-by: Kankan Sarkar <[email protected]>
Co-authored-by: Arcola-Kankan <[email protected]>
Co-authored-by: AlexDLSy <[email protected]>
Co-authored-by: tobiasandorfer <[email protected]>
ebroecker added a commit that referenced this issue Jan 11, 2022
* include LICENSE in source distribution (#365)

* arxml: rx/tx typo fix 

fix for #366

* annotate and fix codestyle dbc (#369)

* Always use Signal.is_little_endian as bool #326 (#371)

* Add coding, remove #!python where inappropriate (#370)

* annotation cleanup ticket #323 (#372)

* Update cli doc (inspired by #361) (#373)

* fix: xls is using the wrong number for arbitration id (#377)

* add outputformat for Scapy (#378)

* add output format for scapy
* fix special char handling in scapy exporter
* add unit test
simplify get_fmt

* Dump factor, offset, min, max as float. Load as numeric (#380)

* Dump factor, offset, min, max as float. Load as numeric

* fixed tests

* fixed tests

* added --jsonNativeTypes as cli option

* review remarks

* ARXML: merge container PDU and Secured PDU support  (#384)

* #381 support for containers
* #381 create signalgroups for containers
* #382 support secured pdus) (#383)
* add unittests for #363 and #382

* Iss381 - fix for #381 - wrong default ArbitrationId handling (#386)

* fix for #381 wrong default ArbitrationId handling

* support for Scapy input file (#387)

* add output format for scapy

* optional ignoring failures durig character encoding #375 (#388)

* fix for #238 (j1939 Frame setters might raise TypeError) (#389)

move j1939 (pgn, prio, source) handling direct to arbitration-id class
(fixes #238)

* Iss385 // Arxml Container support and ARXML refactoring (#390)

ARXML updates:
* fix for #385 
* json fix multiplex reading issue (#136)
* ARXML refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now
fits to next correct dlc value, DBC export reniced for candb++ (#385)
* add test for fit_dlc
* arxml refactoring and better type handling #368
* improvement for #242

* DBC: fix #242. J1939 and FD Frames in dbc

candb seems to be queasy about the order of the VFrameFormat attribute
definition. This Patch seems to generate a dbc, which can contain can_fd
AND J1939-frames.

* XLSX fixes  (#392)

XLSX: fixes #240 (signal init value in excel template)
XLSX: fixes #367 (2.)

* fix for #288 (#393)

* switch from optparse to click (#236) (#394)

* [WIP] add Basic cli tests (#395)

* add tests for global, manipulation and filter swithces of cli.convert
* add basic cli test for cli.compare

* fix some bugs, which were found by these tests

* Annotation cleanup #323 (#396)

* Remove cm prefix (#236) (#399)

Thanks

* add muxing support to scapy output; merge PR #398 also (#400)

* [try to] Unify imports (#402)

* unify imports (#236)

* [WIP] dump/export wireshark lua can subdissector (#404)

* add basic wireshark dissector creator
creastes lua script for dissecting can messages

* add cannelloni dissector for wireshark

* add some doc for scapy and wireshark usage

* [WIP] cycle_time as generic signal and frame attribute  (#405)

* implement cycle_time as generic signal and frame attribute / remove
Gen[Msg/Sig]cycle from attributes  #146
* use effective frame cycle time for export
use gcd of singnal-cycletimes for calculating effective cycle time

This removes one (of many) dbc specific parts from canmatrix and other formats

* Iss407: [dbc] Signal Comments with space bevore semicolon broken import (#409)

* fix for issue #407 [dbc] Signal Comments with space bevore semicolon broken import

* make initial value to a native attribute of Signal class (#408)

* make initial value to a native attribute of Signal class

* [dbf] exended ids corrected

* optionally allow multiple singals with same name in frame #411 (#412)

* Fix setup.py for console_scripts (#417)

* [WIP] [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416)

* optionally allow multiple singals with same name in frame #411
* fixes #413 (space before semicolon mulit line comment)
* fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if
native attribute exists.

* convert: selective rx/tx ecu extraction issue #421 (#422)

* [WIP] [ARXML] can_fd info (potentional fix for #410) (#418)

* potentional fix for #410
* [ARXML] can-fd recogintion
* add baudrate attributes

* [WIP] Auto deploy (#420)

* add auto deploy for tags on master

* add submodules doc

* update doc to make start_bit more clear in documentation #424

* [dbc] fix dbc issues (#431)

* fix dbc export for empty matrix
* copy_frame handles default values now (fix #430)

* fix copy_frame

* fix for issue #436 (#439)

* [convert] add delete_obsolete_ecus (fix for #434) (#438)

* [copy] Iss430 better attribute handling  (#437)

* handling missing default values for defines
* add ecu attribute handling #430

* fix for #441 (#442)

* fix for #441 jsonAll export option working again

* make old test less loud

* Fix some bugs in FIBEX export (#445)

* Fix FIBEX convertion, Handle duplicate signal name

* Repair more click convert parameters (#451)

* Repair command line param names for csv, json, xls(x)

* update tests

* use last supported colorama for pypy3.4

* Fixing issue where we see dictionary attributes change during iteration (#455)

* Arxml limits number formats (#457)

* add number decoding in arxml for different basis
* fix #454

* fix for #460

* fix #460

* Update cli.rst

add arxmlExportVersion documentation  (#461)

* Check for EOF when parsing enums in a .sym file. Fixes #465 (#468)

* Check for EOF when parsing enums in a .sym file. Fixes #465

* Fix test to check for different exception in the python2.7 case

* Log the exception as well when importing a format failed (#467)

* Process the .sym file "Title" field. fixes #469 And Fix Enum dictionary comma splitting  (#470)

* Implement processing of the Title property of .sym files.
* Fix CanMatrix.attribute to return default as documented
* Set the default for the "Title" attribute properly
* Reimplement the quote_aware_comma_split function to handle spaces
between fields
* Change the sym parser to use quote_aware_comma_split to split enum value
tables. Fixes #462.
Add test cases to show that enums are read in correctly from .sym files

* fix for #471 (#473)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474 (#475)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474

Remove the criteria that PF must be >= 240 for PS field to be set as
this seems to be incorrect. (It may be that for some specific case the
PS need to not be used, but it isn't documented here and in general the
PGN should be all 18 bits)
Modifed Tests:

test_canmatrix_get_frame_by_pgn()
seemed to be using made up data and didn't accound for the DP and EDP
fields in expected PGN

test_canid_repr()
Modified the expected repr value to include the PS field in the PGN.

test_canid_parse_values()
Modified expected PGN to include the PS field, Moved the list of
test_data into parametrized cases on a new test.

New Tests:

test_frame_j1939_id_from_components()
Tests constructing a full 29-bit CAN ID from the Priority, Source, and
PGN

test_frame_decode_j1939_id()
Tests extracting Priority Source and PGN from an Arbiration ID

* Add the global desistantion Address 0xFF to the BAM arbitration IDs

* Improve the fix, to more explicitly handle the difference between
PDU-Format 1 and PDU-Format 2, (where the PF >= 240 comes in).
The PGN setter now will now set the arbitration ID to contain the
destination address, but when the PGN getter is used it will strip the
destination field off. I think that it is unlikely for the PGN setter
to be used with PDU-F1 PGN containing a destination. But in the case
that it is I think it makes more sense to not throw the destination
address field away.

* Add enumberations as a field to the json file to store the value tables (#476)

Add a test to check that if a json file is read containing enums and
then saved and re-read it get the same enum table

* Adding changes to allow ArbitrationId classes to be sorted (#466)

* Adding comparators to ArbitrationId
* Changing default Arbitration ID extended to False
* Review based changes
Co-authored-by: Syed Raza <[email protected]>

* Parse sym types (#479)

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Add fix for "isidentifier" fucntion on python 2.7

* Fix logic to match variable name. (Functionality unchanged)

Co-authored-by: Eduard Bröcker <[email protected]>

* Fix sym.dump to export the value_table into the {ENUMS} field of the (#481)

.sym file.
Refactor to remove the use of global variables "enum_dict" and "enums"

Add a test which checks that enums in the original .sym file and enums
which appear as values on a signal are exported to the .sym file.

* Accept types which are in the value table as enum sgnals (#482)

* Fix additional case in sym enum export (#486)

* Accept types which are in the value table as enum sgnals

* Fix previous change to also generate enums from the signal values field
when signal is in a multiplexed frame.

* [DBF] support j1939 read (#501)

Co-authored-by: Broecker <[email protected]>

* possible fix for #490 (#503)

* add tests for issue #424 (#500)

add tests for Motorola forward MSB and Motorola forward LSB start_bit

* Iss484 (#502)

* [arxml] add some basic support for flexray and ethernet data

* extract most of relevant flexray info (#432)

not yet integraded - only extracted by now

* prove of concept for flexray dump (#432)

* starting rework ARXML

* add xlsxwriter to test deps

* add pyyaml to test-reqs

* Update requirements.test.py3.txt

* fix so that tests work again

* remove py3.4 add py3.8

* disable py34 test - enable py38 test

* disable py34

* remove py3.4

* remove py8 warnings

* fix dbc for py8

* fix for #484

buggy ARXMLs with no System-Signals referenced

* fix ci

* fix ci (#504)

* fix ci

* Iss492 (#505)

* integrate fix from #492

* Arxml read baudrate (#452)

* Read Baudrate from arxml

* Enable reading hex values with prefix 0x

* Adaption to pass test with string speed values

* Update to read in hex values define with 0x and integers e.g. 0

* Removal of commented line

* Explicit conversion to string for logger output

Co-authored-by: ForestRupicolous <>
Co-authored-by: Eduard Bröcker <[email protected]>

* Iss496 (#506)

* #496 fix ignore encoding switch

* Iss499 (#507)

* fixes #499

decode_number now supports floating point value

* [WIP] [arxml] add some basic support for flexray and ethernet data (#426)

* [arxml] add some basic support for flexray and ethernet data
* extract most of relevant flexray info (#432)
* starting rework ARXML
* fix dbc for py8
* support for container-pdus with "none" type
* fix for container id byteorder
* fix for secured-ipdus in container
* fix for byteorder big endian header_ids of container-i-pdus
* once again header_id
* Header_id again
* fix for missing compu method
* make SOME/IP at least work a bixt #283
* interprete OPAQUE as intel
* dbc allow '"' in Values
* fix container-pdu without header but with offset
* arxml: fix for container-ipdus, containing same pdu multiple times
* fix enum support for py2.7

* Iss509 (#510)  dbf fix for exporting empty matrix

* dbf fix for exporting empty matrix

* Iss496 (#511)

* #496 fix ignore encoding switch
* clean encoding error implementation

* Master (#514)

* remegerge release (#512)

* Fix sym mux decode, by always treating mux field as unsigned (#517)

* Add a test to show that sym file mux values arent working

* Add a fix to always treat MUX values as unsigned signals

* Log as warning instead of exception (#519)

Logging as exception prints a stack backtrace. Since the modules are optional
and listed as extra instead of a required dependency, this should probably be
a warning instead of an error/exception.

Co-authored-by: Lennart Moltrecht <[email protected]>

* src/canmatrix.py: fix nested mux with no values (#527)

If a complexly multiplexed signal has a sub-multiplexer with no values, a TypeError is thrown.

* add is_fd flag for json output (#532)

Co-authored-by: Dennis Röck <[email protected]>

* Iss526 (#536)

* fix ci

* fix ci

* fixes #526

* fix for #526escaped quotes

* #526 use raw strings for test

* Bump lxml from 4.5.2 to 4.6.2 (#539)

Bumps [lxml](https://github.com/lxml/lxml) from 4.5.2 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](lxml/lxml@lxml-4.5.2...lxml-4.6.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Iss535 (#537)

* fix ci
* possible fix for issue #535

* Iss522 (#530)


* arxml: add reciever information from signals to frames  (issue #522)

* Fix ci (#542)

* fix dbc testcase
* remove xlsx read from tests (#541)

* basic ldf import

* update requirements for ldfparser

* first implementation for odx import#544

* start diag frame at byte 1

* odx service muxer

* odx zwischenstand

* issue #531 Lin Converters

add support for LIN-Converters

* odx zwischnstand

* Revert "odx zwischnstand"

This reverts commit f4476de.

* Update xls.py (#549)

Thanks!

* add direct decode function for python can frames (#548)

* better mux export hanling in sym

* Correctly the result if compare two frame with different ID (#552)

When comparing two dbc, the the frame has different ID it should be marked as "changed" frame.

* Overall code improvements (#555)

* fix: use six.moves instead of future.moves to fix annoying import of top level modules

* fix: missing future import + clean up dependencies

* fix: __package__ does not exist in Python 2.7

* fix: string decoding issue in Python 2.7

* update tox.ini

* fix: install all extras in tox.init

* update travis and appveyor config

* remove unused requirements files

* update tox.ini

* remove unwanted line in test.sh

* fix: Python version 3.9 not available in AppVeyor yet

* fix: only keep minimum requirements for test

* improvement dbc to dbf (#556)

temporary hot fix for canconvert dbc to dbf

* fix issue #547 (#560)

DeprecationWarning: The usage of `cmp` is deprecated and will be removed
on or after 2021-06-01.  Please use `eq` and `order` instead.
    @attr.s(cmp=False)

Signed-off-by: An Nguyen <[email protected]>

* fix issue #559 (#561)

Signed-off-by: An Nguyen <[email protected]>

* fix initial value setting (#566)

Thanks
Co-authored-by: JAZI Nadhmi <[email protected]>

* Set initial_value to physical value (#567)

Currently the initial_value is incorrectly set to the signal raw value.

* arxml fix (#564)

Co-authored-by: Eduard Bröcker <[email protected]>

* arxml.py: (#569)

optimize find/findall
    fix ECU transmitter/receivers for clusters
    rework signals receiers settings

Co-authored-by: JAZI Nadhmi <[email protected]>

* [arxml]: Use ISignal Name for canmatrix.Signal (#570)

Add attributes to  Signal for other names

Co-authored-by: JAZI Nadhmi <[email protected]>

* ARXML Replace ArTree with more simple cache

add option to rename Signal from Signal-Attribute

* add/update doc

* Arxml refactoring (#571)

* ARXML Replace ArTree with more simple cache
* add option to rename Signal from Signal-Attribute
* add/update doc

* rework arxml parsing and use Pdu for PDU contained frames (#576)

Add a converter from PDU contained to multiplexed frame
Add option to handle it in the CLI
Update decoder function

Co-authored-by: JAZI Nadhmi <[email protected]>

* fix convert pdu container (#577)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Remove prints from the code and add the message to the exception (#578)

* remove duplicated pdus (#583)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Improve multiplexer JSON output (#585)

Co-authored-by: Edward Pierzchalski <[email protected]>

* Fix typo (#586)

* bugfix: to get the signals for a pdu you have to use follow_all_ref instead of follow_ref (#591)

Co-authored-by: Markus Konrad <[email protected]>

* improve logging output to be able to identify problems faster/better (#592)

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

* fix for issue #596 (#597)

cope with signals without receiver

Co-authored-by: Eduard Bröcker <[email protected]>

* fix for #403

* arxml init value may not decoded

just ignore #550

* fixes #572

* arxml helpers

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix PDU container decoding (#605)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Kankan patch1 (#604)

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix tox and continuous integration (#609)

* Fixing Issue#610 :: Custom Signal Attributes are removed due to wrong indent

* Kankan patch :: Custom Signal Attributes are removed  (#611)


* Fix #610 :: Custom Signal Attributes are removed due to wrong indent

* #612 Bugfix formats.xls and formats.xlsx missing method parameters (#613)

* Update guess_value method in utils.py (#615)

add a conversion of 0b and 0x string values into int string representation in the guess_value method

initial motivation was to get valid input values for the decimal.Decimal() class
since it crashed in the arxml.py file line 1234:

1232            if initvalue is not None and initvalue.text is not None:
1233                initvalue.text = canmatrix.utils.guess_value(initvalue.text)
1234               new_signal.initial_value = float_factory(initvalue.text)

* Added deleteFloatingSignals , checkFloatingFrames, checkSignalRange, checkSignalUnit, checkSignalReceiver & checkFloatingSignals

* Added J1939 to Extended dbc and vice versa

* feat: add support for Python 3.10 (#619)

* feat: add support for Python 3.10
* chore: fix appveyor configuration
* chore: remove useless commands in appveyor configuration
* chore: Python 3.9 and 3.10 only available for Visual Studio 2019 and up in appveyor

* arxml: read data related to E2E-protection (#594)

* read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object
* also read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object if a Container-PDU is processed

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Bröcker <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Broecker <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Lennart Moltrecht <[email protected]>
Co-authored-by: Ryan Rowe <[email protected]>
Co-authored-by: Dennis <[email protected]>
Co-authored-by: Dennis Röck <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cesare12 <[email protected]>
Co-authored-by: Moris Yun <[email protected]>
Co-authored-by: Darkin <[email protected]>
Co-authored-by: jazi007 <[email protected]>
Co-authored-by: pierreluctg <[email protected]>
Co-authored-by: L X <[email protected]>
Co-authored-by: JAZI Nadhmi <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Ajinkya Pasalkar <[email protected]>
Co-authored-by: tainnok <[email protected]>
Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>
Co-authored-by: Kankan Sarkar <[email protected]>
Co-authored-by: Arcola-Kankan <[email protected]>
Co-authored-by: AlexDLSy <[email protected]>
Co-authored-by: tobiasandorfer <[email protected]>

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Bröcker <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Broecker <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Lennart Moltrecht <[email protected]>
Co-authored-by: Ryan Rowe <[email protected]>
Co-authored-by: Dennis <[email protected]>
Co-authored-by: Dennis Röck <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cesare12 <[email protected]>
Co-authored-by: Moris Yun <[email protected]>
Co-authored-by: Darkin <[email protected]>
Co-authored-by: jazi007 <[email protected]>
Co-authored-by: pierreluctg <[email protected]>
Co-authored-by: L X <[email protected]>
Co-authored-by: JAZI Nadhmi <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Ajinkya Pasalkar <[email protected]>
Co-authored-by: tainnok <[email protected]>
Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>
Co-authored-by: Kankan Sarkar <[email protected]>
Co-authored-by: Arcola-Kankan <[email protected]>
Co-authored-by: AlexDLSy <[email protected]>
Co-authored-by: tobiasandorfer <[email protected]>
ebroecker added a commit that referenced this issue Mar 23, 2022
* merge for release 0.9.4 (#621)

* include LICENSE in source distribution (#365)

* arxml: rx/tx typo fix 

fix for #366

* annotate and fix codestyle dbc (#369)

* Always use Signal.is_little_endian as bool #326 (#371)

* Add coding, remove #!python where inappropriate (#370)

* annotation cleanup ticket #323 (#372)

* Update cli doc (inspired by #361) (#373)

* fix: xls is using the wrong number for arbitration id (#377)

* add outputformat for Scapy (#378)

* add output format for scapy
* fix special char handling in scapy exporter
* add unit test
simplify get_fmt

* Dump factor, offset, min, max as float. Load as numeric (#380)

* Dump factor, offset, min, max as float. Load as numeric

* fixed tests

* fixed tests

* added --jsonNativeTypes as cli option

* review remarks

* ARXML: merge container PDU and Secured PDU support  (#384)

* #381 support for containers
* #381 create signalgroups for containers
* #382 support secured pdus) (#383)
* add unittests for #363 and #382

* Iss381 - fix for #381 - wrong default ArbitrationId handling (#386)

* fix for #381 wrong default ArbitrationId handling

* support for Scapy input file (#387)

* add output format for scapy

* optional ignoring failures durig character encoding #375 (#388)

* fix for #238 (j1939 Frame setters might raise TypeError) (#389)

move j1939 (pgn, prio, source) handling direct to arbitration-id class
(fixes #238)

* Iss385 // Arxml Container support and ARXML refactoring (#390)

ARXML updates:
* fix for #385 
* json fix multiplex reading issue (#136)
* ARXML refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now
fits to next correct dlc value, DBC export reniced for candb++ (#385)
* add test for fit_dlc
* arxml refactoring and better type handling #368
* improvement for #242

* DBC: fix #242. J1939 and FD Frames in dbc

candb seems to be queasy about the order of the VFrameFormat attribute
definition. This Patch seems to generate a dbc, which can contain can_fd
AND J1939-frames.

* XLSX fixes  (#392)

XLSX: fixes #240 (signal init value in excel template)
XLSX: fixes #367 (2.)

* fix for #288 (#393)

* switch from optparse to click (#236) (#394)

* [WIP] add Basic cli tests (#395)

* add tests for global, manipulation and filter swithces of cli.convert
* add basic cli test for cli.compare

* fix some bugs, which were found by these tests

* Annotation cleanup #323 (#396)

* Remove cm prefix (#236) (#399)

Thanks

* add muxing support to scapy output; merge PR #398 also (#400)

* [try to] Unify imports (#402)

* unify imports (#236)

* [WIP] dump/export wireshark lua can subdissector (#404)

* add basic wireshark dissector creator
creastes lua script for dissecting can messages

* add cannelloni dissector for wireshark

* add some doc for scapy and wireshark usage

* [WIP] cycle_time as generic signal and frame attribute  (#405)

* implement cycle_time as generic signal and frame attribute / remove
Gen[Msg/Sig]cycle from attributes  #146
* use effective frame cycle time for export
use gcd of singnal-cycletimes for calculating effective cycle time

This removes one (of many) dbc specific parts from canmatrix and other formats

* Iss407: [dbc] Signal Comments with space bevore semicolon broken import (#409)

* fix for issue #407 [dbc] Signal Comments with space bevore semicolon broken import

* make initial value to a native attribute of Signal class (#408)

* make initial value to a native attribute of Signal class

* [dbf] exended ids corrected

* optionally allow multiple singals with same name in frame #411 (#412)

* Fix setup.py for console_scripts (#417)

* [WIP] [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416)

* optionally allow multiple singals with same name in frame #411
* fixes #413 (space before semicolon mulit line comment)
* fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if
native attribute exists.

* convert: selective rx/tx ecu extraction issue #421 (#422)

* [WIP] [ARXML] can_fd info (potentional fix for #410) (#418)

* potentional fix for #410
* [ARXML] can-fd recogintion
* add baudrate attributes

* [WIP] Auto deploy (#420)

* add auto deploy for tags on master

* add submodules doc

* update doc to make start_bit more clear in documentation #424

* [dbc] fix dbc issues (#431)

* fix dbc export for empty matrix
* copy_frame handles default values now (fix #430)

* fix copy_frame

* fix for issue #436 (#439)

* [convert] add delete_obsolete_ecus (fix for #434) (#438)

* [copy] Iss430 better attribute handling  (#437)

* handling missing default values for defines
* add ecu attribute handling #430

* fix for #441 (#442)

* fix for #441 jsonAll export option working again

* make old test less loud

* Fix some bugs in FIBEX export (#445)

* Fix FIBEX convertion, Handle duplicate signal name

* Repair more click convert parameters (#451)

* Repair command line param names for csv, json, xls(x)

* update tests

* use last supported colorama for pypy3.4

* Fixing issue where we see dictionary attributes change during iteration (#455)

* Arxml limits number formats (#457)

* add number decoding in arxml for different basis
* fix #454

* fix for #460

* fix #460

* Update cli.rst

add arxmlExportVersion documentation  (#461)

* Check for EOF when parsing enums in a .sym file. Fixes #465 (#468)

* Check for EOF when parsing enums in a .sym file. Fixes #465

* Fix test to check for different exception in the python2.7 case

* Log the exception as well when importing a format failed (#467)

* Process the .sym file "Title" field. fixes #469 And Fix Enum dictionary comma splitting  (#470)

* Implement processing of the Title property of .sym files.
* Fix CanMatrix.attribute to return default as documented
* Set the default for the "Title" attribute properly
* Reimplement the quote_aware_comma_split function to handle spaces
between fields
* Change the sym parser to use quote_aware_comma_split to split enum value
tables. Fixes #462.
Add test cases to show that enums are read in correctly from .sym files

* fix for #471 (#473)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474 (#475)

* Fix J1939 PGN getter/setter to use full 18bits. Fixes #474

Remove the criteria that PF must be >= 240 for PS field to be set as
this seems to be incorrect. (It may be that for some specific case the
PS need to not be used, but it isn't documented here and in general the
PGN should be all 18 bits)
Modifed Tests:

test_canmatrix_get_frame_by_pgn()
seemed to be using made up data and didn't accound for the DP and EDP
fields in expected PGN

test_canid_repr()
Modified the expected repr value to include the PS field in the PGN.

test_canid_parse_values()
Modified expected PGN to include the PS field, Moved the list of
test_data into parametrized cases on a new test.

New Tests:

test_frame_j1939_id_from_components()
Tests constructing a full 29-bit CAN ID from the Priority, Source, and
PGN

test_frame_decode_j1939_id()
Tests extracting Priority Source and PGN from an Arbiration ID

* Add the global desistantion Address 0xFF to the BAM arbitration IDs

* Improve the fix, to more explicitly handle the difference between
PDU-Format 1 and PDU-Format 2, (where the PF >= 240 comes in).
The PGN setter now will now set the arbitration ID to contain the
destination address, but when the PGN getter is used it will strip the
destination field off. I think that it is unlikely for the PGN setter
to be used with PDU-F1 PGN containing a destination. But in the case
that it is I think it makes more sense to not throw the destination
address field away.

* Add enumberations as a field to the json file to store the value tables (#476)

Add a test to check that if a json file is read containing enums and
then saved and re-read it get the same enum table

* Adding changes to allow ArbitrationId classes to be sorted (#466)

* Adding comparators to ArbitrationId
* Changing default Arbitration ID extended to False
* Review based changes
Co-authored-by: Syed Raza <[email protected]>

* Parse sym types (#479)

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Allow parsing of the main .sym signal types : string, raw, char.
Add a type_label attribute so that symbol files can be saved back to
.sym format without loss of information.
Add an is_ascii attribute for symbols
Add a check to see if Signal names need Quoting when saving to a .sym
file (if they start with a number or contain a space etc.)

Add a test which reads in a .sym file containing all signal types and
checks that they are all correctly imported and exported.

* Add fix for "isidentifier" fucntion on python 2.7

* Fix logic to match variable name. (Functionality unchanged)

Co-authored-by: Eduard Bröcker <[email protected]>

* Fix sym.dump to export the value_table into the {ENUMS} field of the (#481)

.sym file.
Refactor to remove the use of global variables "enum_dict" and "enums"

Add a test which checks that enums in the original .sym file and enums
which appear as values on a signal are exported to the .sym file.

* Accept types which are in the value table as enum sgnals (#482)

* Fix additional case in sym enum export (#486)

* Accept types which are in the value table as enum sgnals

* Fix previous change to also generate enums from the signal values field
when signal is in a multiplexed frame.

* [DBF] support j1939 read (#501)

Co-authored-by: Broecker <[email protected]>

* possible fix for #490 (#503)

* add tests for issue #424 (#500)

add tests for Motorola forward MSB and Motorola forward LSB start_bit

* Iss484 (#502)

* [arxml] add some basic support for flexray and ethernet data

* extract most of relevant flexray info (#432)

not yet integraded - only extracted by now

* prove of concept for flexray dump (#432)

* starting rework ARXML

* add xlsxwriter to test deps

* add pyyaml to test-reqs

* Update requirements.test.py3.txt

* fix so that tests work again

* remove py3.4 add py3.8

* disable py34 test - enable py38 test

* disable py34

* remove py3.4

* remove py8 warnings

* fix dbc for py8

* fix for #484

buggy ARXMLs with no System-Signals referenced

* fix ci

* fix ci (#504)

* fix ci

* Iss492 (#505)

* integrate fix from #492

* Arxml read baudrate (#452)

* Read Baudrate from arxml

* Enable reading hex values with prefix 0x

* Adaption to pass test with string speed values

* Update to read in hex values define with 0x and integers e.g. 0

* Removal of commented line

* Explicit conversion to string for logger output

Co-authored-by: ForestRupicolous <>
Co-authored-by: Eduard Bröcker <[email protected]>

* Iss496 (#506)

* #496 fix ignore encoding switch

* Iss499 (#507)

* fixes #499

decode_number now supports floating point value

* [WIP] [arxml] add some basic support for flexray and ethernet data (#426)

* [arxml] add some basic support for flexray and ethernet data
* extract most of relevant flexray info (#432)
* starting rework ARXML
* fix dbc for py8
* support for container-pdus with "none" type
* fix for container id byteorder
* fix for secured-ipdus in container
* fix for byteorder big endian header_ids of container-i-pdus
* once again header_id
* Header_id again
* fix for missing compu method
* make SOME/IP at least work a bixt #283
* interprete OPAQUE as intel
* dbc allow '"' in Values
* fix container-pdu without header but with offset
* arxml: fix for container-ipdus, containing same pdu multiple times
* fix enum support for py2.7

* Iss509 (#510)  dbf fix for exporting empty matrix

* dbf fix for exporting empty matrix

* Iss496 (#511)

* #496 fix ignore encoding switch
* clean encoding error implementation

* Master (#514)

* remegerge release (#512)

* Fix sym mux decode, by always treating mux field as unsigned (#517)

* Add a test to show that sym file mux values arent working

* Add a fix to always treat MUX values as unsigned signals

* Log as warning instead of exception (#519)

Logging as exception prints a stack backtrace. Since the modules are optional
and listed as extra instead of a required dependency, this should probably be
a warning instead of an error/exception.

Co-authored-by: Lennart Moltrecht <[email protected]>

* src/canmatrix.py: fix nested mux with no values (#527)

If a complexly multiplexed signal has a sub-multiplexer with no values, a TypeError is thrown.

* add is_fd flag for json output (#532)

Co-authored-by: Dennis Röck <[email protected]>

* Iss526 (#536)

* fix ci

* fix ci

* fixes #526

* fix for #526escaped quotes

* #526 use raw strings for test

* Bump lxml from 4.5.2 to 4.6.2 (#539)

Bumps [lxml](https://github.com/lxml/lxml) from 4.5.2 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](lxml/lxml@lxml-4.5.2...lxml-4.6.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Iss535 (#537)

* fix ci
* possible fix for issue #535

* Iss522 (#530)


* arxml: add reciever information from signals to frames  (issue #522)

* Fix ci (#542)

* fix dbc testcase
* remove xlsx read from tests (#541)

* basic ldf import

* update requirements for ldfparser

* first implementation for odx import#544

* start diag frame at byte 1

* odx service muxer

* odx zwischenstand

* issue #531 Lin Converters

add support for LIN-Converters

* odx zwischnstand

* Revert "odx zwischnstand"

This reverts commit f4476de.

* Update xls.py (#549)

Thanks!

* add direct decode function for python can frames (#548)

* better mux export hanling in sym

* Correctly the result if compare two frame with different ID (#552)

When comparing two dbc, the the frame has different ID it should be marked as "changed" frame.

* Overall code improvements (#555)

* fix: use six.moves instead of future.moves to fix annoying import of top level modules

* fix: missing future import + clean up dependencies

* fix: __package__ does not exist in Python 2.7

* fix: string decoding issue in Python 2.7

* update tox.ini

* fix: install all extras in tox.init

* update travis and appveyor config

* remove unused requirements files

* update tox.ini

* remove unwanted line in test.sh

* fix: Python version 3.9 not available in AppVeyor yet

* fix: only keep minimum requirements for test

* improvement dbc to dbf (#556)

temporary hot fix for canconvert dbc to dbf

* fix issue #547 (#560)

DeprecationWarning: The usage of `cmp` is deprecated and will be removed
on or after 2021-06-01.  Please use `eq` and `order` instead.
    @attr.s(cmp=False)

Signed-off-by: An Nguyen <[email protected]>

* fix issue #559 (#561)

Signed-off-by: An Nguyen <[email protected]>

* fix initial value setting (#566)

Thanks
Co-authored-by: JAZI Nadhmi <[email protected]>

* Set initial_value to physical value (#567)

Currently the initial_value is incorrectly set to the signal raw value.

* arxml fix (#564)

Co-authored-by: Eduard Bröcker <[email protected]>

* arxml.py: (#569)

optimize find/findall
    fix ECU transmitter/receivers for clusters
    rework signals receiers settings

Co-authored-by: JAZI Nadhmi <[email protected]>

* [arxml]: Use ISignal Name for canmatrix.Signal (#570)

Add attributes to  Signal for other names

Co-authored-by: JAZI Nadhmi <[email protected]>

* ARXML Replace ArTree with more simple cache

add option to rename Signal from Signal-Attribute

* add/update doc

* Arxml refactoring (#571)

* ARXML Replace ArTree with more simple cache
* add option to rename Signal from Signal-Attribute
* add/update doc

* rework arxml parsing and use Pdu for PDU contained frames (#576)

Add a converter from PDU contained to multiplexed frame
Add option to handle it in the CLI
Update decoder function

Co-authored-by: JAZI Nadhmi <[email protected]>

* fix convert pdu container (#577)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Remove prints from the code and add the message to the exception (#578)

* remove duplicated pdus (#583)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Improve multiplexer JSON output (#585)

Co-authored-by: Edward Pierzchalski <[email protected]>

* Fix typo (#586)

* bugfix: to get the signals for a pdu you have to use follow_all_ref instead of follow_ref (#591)

Co-authored-by: Markus Konrad <[email protected]>

* improve logging output to be able to identify problems faster/better (#592)

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

* fix for issue #596 (#597)

cope with signals without receiver

Co-authored-by: Eduard Bröcker <[email protected]>

* fix for #403

* arxml init value may not decoded

just ignore #550

* fixes #572

* arxml helpers

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix PDU container decoding (#605)

Co-authored-by: JAZI Nadhmi <[email protected]>

* Kankan patch1 (#604)

* Added option calcSignalMax & recalcSignalMax to calculate max value during converting dbc

* Fixed a wrong keypress mistake

* Removed float conversion before checking if the signal max is zero

* fix tox and continuous integration (#609)

* Fixing Issue#610 :: Custom Signal Attributes are removed due to wrong indent

* Kankan patch :: Custom Signal Attributes are removed  (#611)


* Fix #610 :: Custom Signal Attributes are removed due to wrong indent

* #612 Bugfix formats.xls and formats.xlsx missing method parameters (#613)

* Update guess_value method in utils.py (#615)

add a conversion of 0b and 0x string values into int string representation in the guess_value method

initial motivation was to get valid input values for the decimal.Decimal() class
since it crashed in the arxml.py file line 1234:

1232            if initvalue is not None and initvalue.text is not None:
1233                initvalue.text = canmatrix.utils.guess_value(initvalue.text)
1234               new_signal.initial_value = float_factory(initvalue.text)

* Added deleteFloatingSignals , checkFloatingFrames, checkSignalRange, checkSignalUnit, checkSignalReceiver & checkFloatingSignals

* Added J1939 to Extended dbc and vice versa

* feat: add support for Python 3.10 (#619)

* feat: add support for Python 3.10
* chore: fix appveyor configuration
* chore: remove useless commands in appveyor configuration
* chore: Python 3.9 and 3.10 only available for Visual Studio 2019 and up in appveyor

* arxml: read data related to E2E-protection (#594)

* read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object
* also read informations about E2E-Profiles (of I-Signalgroup) into the canmatrix-object if a Container-PDU is processed

Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Bröcker <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Broecker <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Lennart Moltrecht <[email protected]>
Co-authored-by: Ryan Rowe <[email protected]>
Co-authored-by: Dennis <[email protected]>
Co-authored-by: Dennis Röck <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cesare12 <[email protected]>
Co-authored-by: Moris Yun <[email protected]>
Co-authored-by: Darkin <[email protected]>
Co-authored-by: jazi007 <[email protected]>
Co-authored-by: pierreluctg <[email protected]>
Co-authored-by: L X <[email protected]>
Co-authored-by: JAZI Nadhmi <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Ajinkya Pasalkar <[email protected]>
Co-authored-by: tainnok <[email protected]>
Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>
Co-authored-by: Kankan Sarkar <[email protected]>
Co-authored-by: Arcola-Kankan <[email protected]>
Co-authored-by: AlexDLSy <[email protected]>
Co-authored-by: tobiasandorfer <[email protected]>

* add testcase for Issue #625

* bugfix for #625

* fix is vs. ==

* attributes are more often strings now

Co-authored-by: Daniel Hrisca <[email protected]>
Co-authored-by: Funth0mas <[email protected]>
Co-authored-by: Thomas Fritzsche <[email protected]>
Co-authored-by: chrisoro <[email protected]>
Co-authored-by: Bröcker <[email protected]>
Co-authored-by: Kyle Altendorf <[email protected]>
Co-authored-by: akaanich-technica <[email protected]>
Co-authored-by: Syed <[email protected]>
Co-authored-by: Seneda <[email protected]>
Co-authored-by: Ulf Rüegg <[email protected]>
Co-authored-by: Broecker <[email protected]>
Co-authored-by: Martin Korinek <[email protected]>
Co-authored-by: Leo <[email protected]>
Co-authored-by: Lennart Moltrecht <[email protected]>
Co-authored-by: Ryan Rowe <[email protected]>
Co-authored-by: Dennis <[email protected]>
Co-authored-by: Dennis Röck <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cesare12 <[email protected]>
Co-authored-by: Moris Yun <[email protected]>
Co-authored-by: Darkin <[email protected]>
Co-authored-by: jazi007 <[email protected]>
Co-authored-by: pierreluctg <[email protected]>
Co-authored-by: L X <[email protected]>
Co-authored-by: JAZI Nadhmi <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Edward Pierzchalski <[email protected]>
Co-authored-by: Ajinkya Pasalkar <[email protected]>
Co-authored-by: tainnok <[email protected]>
Co-authored-by: Markus Konrad <[email protected]>
Co-authored-by: Eduard Bröcker <[email protected]>
Co-authored-by: Kankan Sarkar <[email protected]>
Co-authored-by: Arcola-Kankan <[email protected]>
Co-authored-by: AlexDLSy <[email protected]>
Co-authored-by: tobiasandorfer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants