BUG - First Reply Time metrics have mismatches about dbt_zendesk

[fivetran-joemarkiewicz]

Hi @alexei-colvin apologies for the delayed response. Last week was Thanksgiving and I was not able to complete the explanation. I will post back within this Issue and Issue #54 with our DECISIONLOG and also will plan to include the DECISIONLOG.md within the package so others may understand the reason for our transformation logic.

I hope to respond back here by tomorrow with the completed DECISIONLOG 😄

[alanzfu]

Apologies, I just realized that the zendesk__ticket_metrics URL refers to the URL of the ticket, whereas the ticket_metrics.url property from the API refers to the URL of the ticket_metrics JSON. Ignore comment!

[fivetran-joemarkiewicz]

Hi @alexei-colvin thanks so much for opening this issue.

It was great chatting with you the other day and I am looking into this discrepancy as we speak. I will post back here once I have an update and a branch for you to test out. 😄

[fivetran-joemarkiewicz]

Hi @alexei-colvin! I hope you had a great weekend.

I was able to investigate these discrepancies a bit more and found some ticket nuances within Zendesk the package was not accounting for. Would you be able to test the below branch version of the package and let me know if this resolves the calendar first reply times in question?

packages:
    - git: https://github.com/fivetran/dbt_zendesk.git
      revision: bugfix/first-reply-time-update
      warn-unpinned: false

Once you replace your existing version of the package dependency with the version above, you can run dbt deps and then run your zendesk models and identify if the issues still persist. Let me know if have any questions!

[alexei-colvin]

Hello @fivetran-joemarkiewicz I hope you had a great weekend too! Thank you for working in this issue, and for the productive chatting last week!

Regarding this revision, I had tried and yes, the first_reply_time_calendar_minutes metric now match for all the tickets, super! There is still a lot of difference in the metric first_reply_time_business_minutes (more than 1000 minutes for a single ticket). This metric is also very important for our analysis. Are you going to do something regarding this one too?

Thanks for everything.

[fivetran-joemarkiewicz]

@alexei-colvin that's great to hear!

In regards to the first_reply_time_business_minutes I will most likely not (but will let you know if I find anything in my investigations that will result in an update) be updating this metric to reflect the Zendesk UI as our opinion is that this does not accurately capture the true length of time when tickets are exchanging schedules. However, I will put together a write-up on how we have defined our transformation logic so you may see why we have made our decision.

I will include this write-up in the next release, but will also share it here so you may chat about it more before the next release.

[alexei-colvin]

hello @fivetran-joemarkiewicz Thanks! I would really appreciate it if you could explain to me what's going on below, to understand why you think the package's results are better than Zendesk's regarding business times.

[fivetran-joemarkiewicz]

@alexei-colvin thank you again for you patience with me putting together the DECISIONLOG.md for the business time metrics. This was a lot more intensive to document than I was initially intending haha

You can see the full writeup of this decision logic within the DECISIONLOG I have in the development branch. Also, I decided to add a variable that should allow you to override the decision logic and use the Zendesk logic instead if you wish. However, this will not be included as a functionality within the next release of the dbt package and is instead being provided so you may understand the differences between the package output and the Zendesk reports.

To leverage this variable you can add the below config to you dbt_project.yml

vars:
  using_zendesk_business_metrics_logic:    True         #False by default

Let me know if you have any questions when reading through my write-up!

[fivetran-joemarkiewicz]

Hi @alexei-colvin just wanted to check in if you had a chance to read through the DECISIONLOG above and if it helps answer your question about the business metrics differences between the dbt package and Zendesk?

[alanzfu]

@fivetran-joemarkiewicz would this same logic also potentially affect the ticket_metrics URL as well?

[fivetran-joemarkiewicz]

Hi @alanzfu 👋 Would you be able to elaborate a bit more on how the ticket_metrics URL?

[fivetran-joemarkiewicz]

Closing this issue as the merged PR above addresses the issue identified. Please feel free to open the ticket if this issue still persists.

[calder-holt]

@fivetran-joemarkiewicz we stumbled upon this thread because we've identified the same behavior the issue targets.

I see you mention a variable that was not added to a future release of using_zendesk_business_metrics_logic, is that still the case and is there a way to override the package to use the zendesk logic? We are having a hard time finding any reference to the variable in y'alls code base or on the branch mentioned above.

[fivetran-avinash]

Hi @calder-holt ! The variable appears to have been introduced for testing purposes only and was later removed in a later release after tests were validated. You can see in our DECISIONLOG our reasoning for deciding not to capture this data in the same way the Zendesk UI captures it, leading to the discrepancies you are seeing.

To override the package to use the zendesk logic, I'd take a look at this post on how to override the logic of one of our models while still leveraging the package as a whole. This should allow you to make the adjustments you need to achieve your desired outcome.

Let me know if you have any questions!