data-dog / datadogauditbundle Goto Github PK

Hi,

Is it possible to forward the logging to queue system (for example rabbitmq)? I'm afraid that flushing the logs on the same transaction will slow it down. By forwarding it to queue system on post flush, it will fasten the process while still have the logging feature.

Thanks.

The class 'DataDog\AuditBundle\Entity\AuditLog' was not found in the chain configured namespaces

facing this issue

Hello,

I'm facing this error on entity edit :

Here is SF configuration :

bundle config :

data_dog_audit:
    audited_entities:
        - App\Entity\Formation

Tell me if you want more informations about the "bug".

Currently unable to install this bundle with Symfony 5.

Hello
I can't run the demo
Because there are a lot of issues when trying make

This is an awesome bundle! Only issue I'm having is that my project bundle is called AppBundle and so is the one in the example. Which means that my Project and User entities get namespace conflicts with the example.

It doesn't actually cause any errors, but it does make PhpStorm complain that there are multiple definities for the classes.

Is there a way I can install the bundle without the example?

Add a repository for fetching entries.

There is no documentation how various relations are stored.

Please write documentation about this.
Or even better write the OneToMany associations handier. (Doing database queries for a json string is unhandy and not precise. Even finding the fk requires this hack.)

What I found out:

ManyToMany association, owning side:

AuditLog (of owning side entity):
   action: 'associate' #
   source:
       # -> points to Association
       id: # internal id of this association
       typ: # type (similar to bundle+class) of associated entity
       tbl: # table where the entity of this association is stored
       label: # value of associated entity, as string
       fk: # id of the associated entity
       class: # class of associated entity
   target:
       # -> points to Association
       class: class of related association
       tbl: table of ...
       fk: id of related association
   diff: []
       # empty array

ManyToMany association, other side:
#nothing

OneToMany association, owning side:

AuditLog (of owning side entity):
   action: 'update' # or insert
   source:
       # -> points to Association
       fk: id of this audit log
   target:
       null
   diff: # as json
       ...
       associationProperyName:
           old:
               typ: # see in ManyToMany 
               tbl: # see in ManyToMany 
               label: # see in ManyToMany 
               fk: # id of the associated entity
               class: # see in ManyToMany 
           new:
               typ: # see in ManyToMany
               ...
           col: # column name

Has anyone done any work on purging data for a specific entity and all its relations.

ie, we store data for a fixed period of time, after which we purge the "case" and need to remove all its audit records including a fairly deep set of nested relations.

not everything is stored for the same period, so it's not just a case of deleting old entries - the cut off is defined by a specific action on each case which is extremely variable

We have a bunch of entities with title fields that relate to the title of the user (Mr, Mrs, Miss)

Your switch statement on the subscriber prioritises a title field, which means the title gets saved as Mr rather then our __toString or name fields.

If you swap __toString to be the upmost priority (as toString is the defacto string representation of the object not any other field) and then check for other fields if its not implemented it would make it more user friendly. Adding a config in the yml for selecting a label field would also be a better implementation.

I have overridden the subscriber, created a new subscriber, inherited and overridden the label function myself for our project, but I think it would be beneficial to swap the order in the repo. I can make a pull request for this if you don't have time.

Other then that its a great bundle. Thanks.

Hello,

This leads to

Fatal error: Uncaught Symfony\Component\Debug\Exception\FatalThrowableError: Maximum function nesting level of '256' reached, aborting! in /srv/vendor/monolog/monolog/src/Monolog/Processor/PsrLogMessageProcessor.php on line 31

Symfony\Component\Debug\Exception\FatalThrowableError: Maximum function nesting level of '256' reached, aborting! in /srv/vendor/monolog/monolog/src/Monolog/Processor/PsrLogMessageProcessor.php on line 31

when a one is flushing more than 256 times in a script, while using this bundle. One can see in the stack trace a long chain of LoggerChain.php instances, which at some point trips the php's configured max recursion level.

There are use cases when one needs to flush a lot in a script.

Fyi.

Currently a blame entry is written to the audit_association table for every changed table. (Example: when one table and two associated ones are changed, the blaming user is written 3 times with the same info.) All of this blame entries contain the same data. (Because DataDogAudit is called at the very end, the user entity can not change while writing the log.)
Writing the same data multiple times bloats the database unnecessary.
This should not break backward compatibility (reading is the same as before).

ToDo:

• Adapt the entity: ManyToOne instead of OneToOne for blame (does not change the database)
• write blame only once in one flush() call, all written AuditLog entries point to this one.

Hi There,
I've looked through the documentation, and I can't see any configuration or similar in regards to the archival or rotation of entries.
The reason being that the audit tables get pretty large quite quickly on one of our instances.
Is there anything baked-in in terms of cleanup?
Thanks

Currently this bundle isn't compatible with Symfony 6.

Greetings,

An exception is thrown when trying to use the bundle with ldap authentication.

Ldap user provider uses symfony in memory user class Symfony\Component\Security\Core\User\User. This class has no Id field.

Can please take a look at it

Regards,

I've recently run into a problem, we need to do a database update which requires some data migrations. I've written a symfony command to do the data migrations but it creates an AuditLog with a null blame id. I was able to get around the problem my hacking around it and simulating a login like this: https://stackoverflow.com/questions/22606444/symfony-2-simulate-login-and-security-context-in-console-command

But it would be great if we could programatically set the user that the bundle will use, just like the doctrine extensions blameable: https://github.com/Atlantic18/DoctrineExtensions/blob/v2.4.x/lib/Gedmo/Blameable/BlameableListener.php#L60

I'm happy to contribute this myself if you would like?

Looks like it could be an unintended consequence of the mapping changes in previous commits, and may only affect postgres, I'm not using this bundle with any other DB platforms at the moment so unable to confirm if it happens on Mysql, SQLite etc ...

(https://user-images.githubusercontent.com/4586381/157653353-4d1724d1-9249-46e6-aa12-ce9234b9ba86.png) exception

Originally posted by @acornforth in #93 (comment)

I can't get any soft-delete event logged. I'm using DoctrineExtensions for soft-delete but changes doesn't get logged as remove and neither as update.

Drop support for unsupported versions of Symfony and Doctrine. This will reduce complexity with backwards compatibility.

• Remove yaml loader dependency

TODO:

• Automated Testing (#86)
• Drop unsupported package versions (#87)
• Table indexes (#88)
• Repository to get entries (#89)

Is it possible to change the name of the tables and fields that are created by the package?

Is there any possibility to retrieve the ID of the entity which is created/updated/deleted? In the audit_logs table I only see the fields that are inserted in the 'diff'-field but not the ID's such that I can retrieve the particular entity.

Hello, this is a nice bundle and I want to implement it inside my project.
I'm using Symfony 3.3. I just include your src bundle and it's saving the data. But I want it to show the data and I don't know how to.
Another thing is that I think you´re not saving the user that's make the changes, is that right?
Thanks a lot for this bundle and I hope you can help me with it.
Best

Add automated tests to ensure compatibility with supported package versions.

In the audit_associations table, Doctrine proxies are being used for the class and typ columns instead of the actual entity.

Expected:

Result:

Currently every log entry gets its own timestamp. This is the same except when the second changes while the log is written (rounding to seconds).
This difference in the timestamp is an irrelevant detail. (Writing the entity has happened before anyway.) And it makes reading more complicated. Therefore please write the same timestamp while in one flush call.

Hi,

I am using your auditing bundle and I seem to have encountered a problem...

Here is the situation

I have a doctrine entity with a inverse one to one relationship.
When I try to modify it, auditing triggers and I get the following error

'MyProject\MyBundle\Entity\EntityA#entityB' is not an association with a single join column.

The error is triggered on line 377 of 'DataDog\AuditBundle\EventSubscriber\AuditSubscriber' :

private function diff(EntityManager $em, $entity, array $ch)
{
    $uow = $em->getUnitOfWork();
    $meta = $em->getClassMetadata(get_class($entity));
    $diff = [];
    foreach ($ch as $fieldName => list($old, $new)) {
        if ($meta->hasField($fieldName)) {
            $mapping = $meta->fieldMappings[$fieldName];
            $diff[$fieldName] = [
                'old' => $this->value($em, Type::getType($mapping['type']), $old),
                'new' => $this->value($em, Type::getType($mapping['type']), $new),
                'col' => $mapping['columnName'],
            ];
        } elseif ($meta->hasAssociation($fieldName) && $meta->isSingleValuedAssociation($fieldName)) {
            $mapping = $meta->associationMappings[$fieldName];
            $colName = $meta->getSingleAssociationJoinColumnName($fieldName);
            $assocMeta = $em->getClassMetadata($mapping['targetEntity']);
            $diff[$fieldName] = [
                'old' => $this->assoc($em, $old),
                'new' => $this->assoc($em, $new),
                'col' => $colName,
            ];
        }
    }
    return $diff;
}

The else if block checks for $meta->isSingleValuedAssociation($fieldName) before calling $meta->getSingleAssociationJoinColumnName($fieldName); but in the Doctrine ClassMetadataInfo class we see :

public function getSingleAssociationJoinColumnName($fieldName)
{
    if ( ! $this->isAssociationWithSingleJoinColumn($fieldName)) {
        throw MappingException::noSingleAssociationJoinColumnFound($this->name, $fieldName);
    }
    return $this->associationMappings[$fieldName]['joinColumns'][0]['name'];
}

The function calls isAssociationWithSingleJoinColumn (which is not checked for in the bundle code) and throws an exception.

I am using doctrine version 2.4.8.

Thanks!

You have declared the AuditLog's field "diff" as "json_array" type, for which doctrine handles the conversion automatically.
In AuditSubscriber you convert to json the diff data (here and here) and the result in the database is that we have data encoded twice (in my case the second conversion applies a \ before every ") and when we try to retrieve the data using getDiff() we've the data as a json and not as array.
We are doing something wrong?

We're using Symfony 3.4 with doctrine/dbal 2.5

Update
We've seen pull request #51, and we think that it might've been implemented by mistake (probably it was an encoding issue on Mopster's side, since doctrine's json and json_array type require UTF-8 encoding on the DB's column).

Is it possible to have this bundle work for multiple connections?

I have the Users table in a separate database, and I need to keep an audit log of what the users are changing in different databases.

I see that the "assocations" table only keeps string values so it should be possible but I installed it, configured it in each connection in doctrine.yaml, but it does not store audit for cross-connection changes.

I see the setLabeler function in the audit subscriber but I don't know where to set it. Could you provide an example of how to use the labeler in the example?

Add indexes for columns that are likely going to be used for searching for data.

Meaning keep track of modification and revert them with a version number ?
(I am testing the loggable tool to do so, but I am interesting to use your tool as well to track every changes)

Thanks for your work !

@l3pp4rd Do you have an idea if there are issues with making this compatible with Symfony 3?

Hi!

I'm looking to use this bundle, but found OneToMany associations are not recorded on inversed side (mappedBy) of a bi-directional relation.

This makes it hard to see what relations an entity had on creation.
You can see it from owning side, but this isn't really enough for a good audit.

For example If you have Project with OneToMany Tickets, Ticket->Project is recorded when viewing the diff for a ticket, but Project->Tickets is not recorded when viewing the Project, and it's very hard to find the tickets for a project.

I've had a quick look at the code. We could approach it two ways.

1. use your AuditSubscriber::associate method for One-To-Many relations (at the moment it only does Many-To-Many collection changes).
2. Record the collection in the diff, so on the entity diff page you would see a link to all Tickets for a Project (just like on the Ticket diff page you see a single link to a project).

Possible option 1 might be nicer, as dissociate could also be used to record removals, for example if a Ticket was deleted, a Project would be dissociated from the ticket.

Would you be willing to accept a PR?
Do you have a preference on approach?

Thanks.

When logging in as user Yoda, I get
You have requested a non-existent service "security.context".

When extending AuditSubscriber.php in order to override the blame() method, static analysis tools (phpStan) complain about unreachable code since it thinks that $blameUser cannot be null.

Typehint annotation for the $blameUser prop, should be /** @var UserInterface|null $blameUser */

better yet, since support for php <7.4 is generally being dropped all over the place, maybe it's time to start using some proper PHP native types ?

PHP Fatal error: Uncaught TypeError: Argument 1 passed to DataDog\AuditBundle\EventSubscriber\AuditSubscriber::__construct() must be an instance of Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorage, instance of Symfony\Component\Security\Core\Authentication\Token\Storage\UsageTrackingTokenStorage given, called in /tmp/intima_admin/cache/ContainerKRKwjFo/srcApp_KernelDevDebugContainer.php on line 373 and defined in /home/vagrant/code/intima_admin/vendor/data-dog/audit-bundle/src/DataDog/AuditBundle/EventSubscriber/AuditSubscriber.php:49

Hi,
We're currently building a new large project with a relatively large database (around 200 tables).
We need to keep track of changes done on mostly all tables and be able to show diffs through the UI.
This bundle seems to fit our needs but, if I'm not mistaken, it stores changes in only two tables. So, with a lot of users, I fear that we'll quickly have two gigantic tables with billions of records which will make them slow to lookup or to query (to show diff in the UI) and as a consequence will be harder to backup (and restore).
So, is it possible to split audit logs into more tables (one audit table per audited table for example as simplethings/entity-audit-bundle does) ?
If no, do you have some feedback about similar use cases?

As in subject.

Example on defining labeler (as from docs):

    datadog.event_subscriber.audit:
        class: 'DataDog\AuditBundle\EventSubscriber\AuditSubscriber'
        arguments: ["@security.token_storage"]
        tags:
          - { name: doctrine.event_subscriber, connection: default }
        calls:
            - ['setLabeler', [['\AppBundle\Labeler\AuditLabeler', 'getLabel']]]

debug:config sees config okay
dump inside isEntityUnaudited proves the array of unaudited entities is empty.

Hi,
is there any plan to support setting blamed user from impersonated user (where applicable)?

Are PR accepted?

Cheers

I don't think it should create multiple rows with same data whenever object is used?

Hello,
I think about possibility not to log specified fields on audited entity. We could use for each variable representing column (in Entity) such annotation: @DataDogAudit\Unaudited . These information can be used to filter $uow->getEntityChangeSet output (unset unaudited elements in array). Due to that change, we can for example avoid logging critical private data or columns changed automatically by application. If I would develop this functionality, would it be included in audit bundle? (changes in AuditSubscriber, probably new file in DBAL folder) .

don't work with composite key ?

I want to log all changes of ORM entity into a Mongo database. Is this possible?

https://www.doctrine-project.org/projects/doctrine-orm/en/2.6/tutorials/embeddables.html

There should be support for it or some ability to skip it, because now it will fail to log whole entity.

I'm running into an exception thrown in the DataDog\AuditBundle\EventSubscriber\AuditSubscriber class, line 320 when the subscriber tries to record changes to an entity with a composite key.

For now, I'm just adding it to unaudited entities. I'm willing to do modifications to help allow composite key entities to be audit-able. I'm thinking there may some discussion needed on strategy for handling things. I'll have to look into it more thoroughly when I find a space of time.

I installed the bundle, I checked that it registers in database, but I do not know how to show the view with logs.