nebo15 / sage Goto Github PK

I try to build an example app with Sage.

I have something like this:

  def purchase() do
    Sage.new()
    |> Sage.run(:order, &order_effect/2, &order_compensation/4)
    |> ...
    |> Sage.execute(%{})
  end

  defp order_effect(_effects_so_far, _attrs) do
    Supplier.order() # let's assume this keeps retunring {:error, :no_response}
  end

  defp order_compensation(_effects_to_compensate, _effects_so_far, _stage_error, _attrs) do
    {:retry, retry_limit: 2}
  end

So, if the first stage in my saga errors, I want to retry 2 times. After 2 retries, if it keeps erroring, I want to log the error and then :abort

If I would not use the :retry I could simply do

  defp order_compensation(_effects_to_compensate, _effects_so_far, _stage_error, _attrs) do
    Logger.error("my error")
    :abort
  end

But I don't know how to handle this if I want to keep the retry option.

I was thinking to introduce some kind of catch_all stage as first stage, but not sure at all if that is the correct approach. Something like:

  def purchase() do
    Sage.new()
    |> Sage.run(:catch, &no_effect/2, &catch_compensation/4)
    |> Sage.run(:order, &order_effect/2, &order_compensation/4)
    |> ...
    |> Sage.execute(%{})
  end

  defp no_effect(_effects_so_far, _attrs) do
    {:ok, %{}}
  end

  defp catch_compensation(_effects_to_compensate, _effects_so_far, {:order, :no_response}, _attrs) do
    Logger.error("my error")
    :abort
  end

Also, as a side question, what would be the difference between returning :ok and :abort in the compensations? In my example, they behave the same so far.

Thanks so much!

MFA's are rather ugly and I'm not sure why someone would use them, resulting code is harder to read. We might build a Sage.Operation behavior with two callbacks:

# required
@callback transaction(effects_so_far :: effects(), execute_opts :: any()) ::  {:ok | :error | :abort, any()}

# optional
@callback compensation(effect_to_compensate :: any(),
           effects_so_far :: effects(),
           execute_opts :: any()) :: :ok | :abort | {:retry, retry_opts :: retry_opts()} | {:continue, any()}

Introducing this change and removing MFA would help to make sure that saga can be type checked #17.

First, thank you for this lib, we use it in quite a few places ❤️

We recently tried upgrading from 0.4 to 0.6 and we see lots of test failures. Even if Sage is still pre 1.0, I think it would be good to keep a CHANGELOG and note breaking changes. As it is, we'll have to comb through a lot of commits to see what changed between 0.4 and 0.6, as those are quite some time apart.

Thanks,
Mihai

I have few sagas which can be used independently and in another more high-level sagas. Just curious what is the proper way to use shared arguments/stages with such workflow.

Right now I have two functions process(%Sage{} = sage) which running all stages and run(...). So I assume required data always in effects_so_far. For run function I pass accepted arguments into "fake" stages:

Sage.new()
|> Sage.run(:required_effect, fn _effects, _params -> {:ok, arg1} end)

What is proper way to do it?

The idea came from Azure Architecture guidelines:

Placing a short-term timeout-based lock on each resource that's required to complete an operation, and obtaining these resources in advance, can help increase the likelihood that the overall activity will succeed. The work should be performed only after all the resources have been acquired. All actions must be finalized before the locks expire.

Hey 👋

When you click the view source button on Hexdocs:

It links to this URL which is broken https://github.com/Nebo15/sage/blob/v#{@version}/lib/sage.ex#L1

I would fix but I don't know how 🙈 . Maybe updating hexdocs?

Hi - first off let me say that sage is super cool! I've been using it in my project for a few weeks now and it REALLY clears things up.

One small thing that had me puzzled was that the readme examples all talk about fallbacks having /3 arguments:

 |> run(:plans, &fetch_subscription_plans/2, &subscription_plans_circuit_breaker/3)
 |> run(:subscription, &create_subscription/2, &delete_subscription/3)

which gives you a compile error, as the code currently requires /4 :

effect_to_compensate, effects_so_far, name_and_reason, attrs

Would it be okay if I wrote a PR for this?

Thanks again! Bart

Dialyzer should check the types of the composing calls, so if two funs don't match up the success typing can find it.

https://hexdocs.pm/phoenix/1.3.2/Phoenix.Endpoint.html#module-callbacks-cycle

I believe Ecto.Multi allows this. It would be useful for cases in which you're doing the same operation multiple times. For example:

users = get_users()
Enum.reduce(users, Sage.new(), fn user, sage ->
  Sage.run(sage, {:promote_user, user.id}, &promote_user/2)
end)

or anything along those lines.

SAGE 0.4, after a compensation runs, the transaction is being passed effects_so_far as expected:

iex(1)> (Sage.new()
...(1)> |> Sage.run(:invoice, fn _, _ -> {:ok, "invoice"} end)
...(1)> |> Sage.run(
...(1)>   :some_step,
...(1)>   fn a, b ->
...(1)>     IO.puts "some_step called with a=#{inspect(a)} b=#{inspect(b)}"
...(1)>     if :rand.uniform(10) >= 3 do
...(1)>       IO.puts "error"
...(1)>       {:error, "some_error"}
...(1)>     else
...(1)>       IO.puts "ok"
...(1)>       {:ok, "some_result"}
...(1)>     end
...(1)>   end,
...(1)>   fn _, _, _, _ ->
...(1)>     {:retry, retry_limit: 10, base_backoff: 1, max_backoff: 1, enable_jitter: false}
...(1)>   end
...(1)> )
...(1)> |> Sage.transaction(Ypsilon.Repo))
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{invoice: "invoice"} b=[]
ok
{:ok, "some_result", %{invoice: "invoice", some_step: "some_result"}}

SAGE 0.6, after a compensation runs, the transaction is being passed an empty effects_so_far

iex(1)> (Sage.new()
...(1)> |> Sage.run(:invoice, fn _, _ -> {:ok, "invoice"} end)
...(1)> |> Sage.run(
...(1)>   :some_step,
...(1)>   fn a, b ->
...(1)>     IO.puts "some_step called with a=#{inspect(a)} b=#{inspect(b)}"
...(1)>     if :rand.uniform(10) >= 3 do
...(1)>       IO.puts "error"
...(1)>       {:error, "some_error"}
...(1)>     else
...(1)>       IO.puts "ok"
...(1)>       {:ok, "some_result"}
...(1)>     end
...(1)>   end,
...(1)>   fn _, _, _ ->
...(1)>     {:retry, retry_limit: 10, base_backoff: 1, max_backoff: 1, enable_jitter: false}
...(1)>   end
...(1)> )
...(1)> |> Sage.transaction(Ypsilon.Repo))
some_step called with a=%{invoice: "invoice"} b=[]
error
some_step called with a=%{} b=[]
ok
{:ok, "some_result", %{some_step: "some_result"}}

The use case is described in #50

Imagine an app called Blog that uses Sage.

Currently Sage allows async steps that spin up a task supervised under Sage.AsyncTransactionSupervisor. Because Sage is a dep of Blog, on shutdown Blog will shutdown before Sage does.

That means if an async step interacts with a process under Blog's supervision tree and you are using some sort of rolling deploy you can enter the following scenario:

Blog starts async task
Sage starts doing that task
Blog shuts down
Sage tries to interact with a process in Blog and can't - so it crashes / errors.

Possible Solutions

Make Sage a library and allow users to start it themselves?
Allow the user to pass in their own supervisor under which it can start the async steps.

This PR does 2

Allow async operations to optionally specify dependency after which they want to run:

sage
|> run_async(:a, tx_cb, cmp_cb)
|> run_async(:b, tx_cb, cmp_cb, after: :a)
|> run_async(:e, tx_cb, cmp_cb)
|> run_async(:c, tx_cb, cmp_cb, after: [:b, :e])

To implement this we need a run-time checks for dependency tree to get rid
of dead ends and recursive dependencies before sage is executed.

I am just trying Sage for the first time and therefore not sure if I am using it correctly.

When running a saga, its return value looks like: {:ok, last_effect, effects}. Wouldn't it make more sense to just return {:ok, effects}. As from a consumer's point of view I do not want to know about which step was last and what it's result was.

And where would I format / post process it. Would you put a case statement right after a saga's steps or where the saga is actually called?

https://github.com/Nebo15/sage/blob/master/lib/sage/executor/retries.ex#L32

In some specific cases I have to run sequences of sub-sages in one high-level sage.

run() allows to attach sages perfectly but only in forward direction. If error occurs somewhere in sub-sages I need to compensate everything happened before the error.

After the sage is completed, I'm left only with its effects and it would be nice to have some rollback(sage,effects) method to provide compensations mech for sub-sages.

Now I have to build nested sages with a last run() in each sage to run the subsequent sage, which seems an overkill for such a straightforward requirement.

Hi!

Do I understand correctly that even if I call Sage.transaction in my saga, my SQL changes in run_async stages are not going to be rolled back due to the fact, that repo.transaction/2 and repo.rollback/1 are executed in the 'main' process while run_async step is executed in another process via Task.async? My understanding is that the task process would checkout new connection from ecto pool, run the queries and that's it. The transaction/rollback function calls are happening outside of the async process within different database connection, therefore the changes made in async stage are not going to be rolled back.

Thank you for your time!

I'll keep this list updated for future readme overhaul:

• How to act when the side effect is updated (https://elixirforum.com/t/re-sage-sagas-implementation-in-pure-elixir/28498?u=andrewdryga)
• How and where attrs should be validated so that return struct would look like input? (Ecto.Changeset for all attrs alltogether either before Sage or as one of it's steps, should we add a helper for that?)
• Avoid using names in functions, keeping them pure (eg. instead of run(:foo, &bar/2) use run(:foo, &bar(&1.fiz, &2)) so that transaction functions are pure and do not depend on each other.

• Expand example to be an actual module with a sample implementation of functions (to show callbacks and what they are doing)
• Demonstrate Sagas composition
• Write best practice on Sagas testing

See https://gist.github.com/michalmuskala/5cee518b918aa5a441e757efca965d22.

HTTP client can extend Saga's interface and we will be able to:

• suggest the way to compensate request;
• require compensation for non-idempotent operations;
• reduce amount of code that needs to be written for complex API interactions.

I often use tuples for stage names, especially when performing sage transactions in a loop.

The inspect implementation raises an ArgumentError when stage name is a tuple.

Should be a simple fix. I'll open a PR next weekend if nobody else picks it up by then.

lib/sage/inspect.ex:22

  defp format_stage({name, operation}) do
    name = "#{Atom.to_string(name)}: "
    group(concat([name, nest(build_operation(operation), String.length(name))]))
  end

Just curious if we can introduce API to compose sagas. For example in our rental application process we have such sagas:

1. application form processing - add row into db, charge fee, generate PDF, send emails
2. credit report processing - based on form we sending data into Credit bureau, generating PDFs, etc. adding info to db, etc.
3. rental deposit processing - add info to db, charge deposit amount, etc.

These 3 steps in it's own saga since we need to do it independently (eg. ask for credit report again after it was failed on credit bureau side).

But we have high-level super-saga (we name it ApplicationSaga) which runs all this 3 sagas and something more.

Right now we have 2 functions in each saga - run which runs saga independently and process which accept %Sage{}, piping stages and returns it.

But I am curious if it will make sense something like

Sage.run_sub(sage, :credit_report, another_sage, ...). Notice there is another_sage instead of function as 3rd argument.

Basically my idea is to:

1. isolate stage names and params between this sagas when it's used as part of another sagas
2. isolate saga results. In my "main" saga I don't care about each stage result from sub-saga. I just need final result provided by sub-saga.

Hi,

I wonder if you need to have all the required data to run a saga upfront.

If I use the flight booking example, after having the users chose they seats, we would need to ask them if they want to rent a car, then if they want to book a hotel room, and then which kind of room, etc.

This cannot be executed in the same code block, we need to get the user input (for instance, we have a website and we display the current state of the transaction, ask questions, and get HTTP requests as the input).

But looking at the examples it seems that you need to have any required data available before calling Sage.transaction ; and also that unless this function is called you do not run any of the intermediary steps. So:

• to book a car you need to know if the user wants a car
• to know if the user wants a car you need to pre-book the flight first (and that needs to be compensated if the booking is cancelled)
• to pre-book the flight you need to run the transaction
• to run the transaction you need to know if you have to book a car

Is there an example that addresses this problem?

Thank you

Hello,

While I really like how the library looks, I think it would be nice to have a more precise description of the guarantees that it provides.

For example:

Sage guarantees that either all the transactions in a saga are successfully completed or compensating transactions are run to amend a partial execution.

I think we all should be really careful when using words like "guarantees" in case of distributed systems because it's really easy to provide a false sense of confidence.

I think it would be really good to consider following example situations:

1. What if a process executing a pipeline fails?
2. What if node is restarted during the pipeline?
3. Can the pipeline be retried by a different node in a cluster?
4. What if external API fails and it's impossible to revert a step?
5. What if a request to the external API actually succeeds, but the reply is lost or there's a timeout?

Note that it isn't a criticism of the library. I know that these are all really hard problems and I don't expect they will be solved by the library (some of them cannot be solved anyway), I just think it's important to provide a more formal section on guarantees so the clients are aware of the potential problems and risks.

Of course I'd be happy to with the description (if I'm competent enough) :)

Thanks!

Does Sage work with Commanded or some other CQRS / Event-Sourcing Framework?

Environment

• sage version: 0.6.1
• Elixir & Erlang/OTP Versions (elixir --version) Elixir 1..13.1 / OTP 24

Current Behavior

I created a repo https://github.com/sparkertime/sage_exceptions_oddity where I can reproduce what seems like strange behavior to me. If a transaction raises an exception during run_async, it still dumps the exception to the terminal even though the process continues.

In that repo, I define two functions

  def bust_synchronously() do
    Sage.new()
    |> Sage.run(:foo, &this_errors/2)
    |> Sage.execute()
  end

  def bust_asynchronously() do
    Sage.new()
    |> Sage.run_async(:foo, &this_errors/2, :noop)
    |> Sage.execute()
  end

  defp this_errors(_, _), do: raise(RuntimeError, "boop")

Both of these tests pass, but the bust_asynchronously test produces this console output before passing

14:44:06.800 [error] Task #PID<0.216.0> started from #PID<0.215.0> terminating
** (RuntimeError) boop
    (sage_exceptions 0.1.0) lib/exceptional_saga.ex:14: ExceptionalSaga.this_errors/2
    (sage 0.6.1) lib/sage/executor.ex:212: Sage.Executor.apply_transaction_fun/4
    (elixir 1.13.1) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.13.1) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: #Function<1.58992025/0 in Sage.Executor.execute_transaction/4>
    Args: []

We can add a persistence adapter for Sage. The main goal of implementing it is to let users recover after node failures, making it possible to get better compensation guarantees. As I see it we need:

1. Ability to use adapters (probably Ecto and RabbitMQ) to persist execution stages;
2. Callbacks/Behaviour to define loader that resumes persistent compensations that are not completed.
3. We need to make sure that compensations are processed "exactly once", or "at least once" (we need to take a tradeoff here).

This mechanism should be prone to network errors, eg. we should persist a state before executing a transaction, so that even if the node is restarted, compensation (even trough without knowing anything about the effect) would still start, potentially on another node.

It would make logging so much nicer 🌮

Hey! I'm super pumped to see this upgrade as I know a lot had built up in master.

While I'm excited to see that compensations should no longer have knowledge of failed steps, there are quite a few compensations in my old code where I was doing something like this:

def revert(_sc, effects, {:raise, _exception}, _attrs), do: ...

i.e if something were to raise downstream, I still have to compensate.

I think I can still act on this by checking at all the effects so far and an absence of a given stage probably means it either didn't get there or that stage raised? Feels brittle - just curious if you have a recommended pattern here that you had in mind when removing this info from the compensations.

Thanks!

Similar to elixir-ecto/ecto#3565

Hello.

On our current app we have resuable "Tracer" module where can trace things, collect metrics, and etc.
But we would like to implement a way for developers to opt in or opt out some of these features.

So for example:

with_tracer(sage, MyApplicationTracer)

We would like to do something like:

with_tracer(sage, MyApplicationTracer, metrics?: true, trace_span: false, logging: false)

Today that is achievable by creating multiple modules:

TraceWithMetrics
TraceWithMetricsAndSpan
TraceWithMetricsButNotSpanWithLogging

Obviously not scalable. We could use the state/context, on execute:

execute(sage, %{my_thing | tracing_opts: [metrics?: true, trace_span: false, logging: false]}

The problem of that approach is limit the things that we can use in execute to be always a map. Approachable, but not great.

Another option is similar to module one, but using macros

def MyApp.TracerBuilder do
  defmacro ___using___(opts) do
#...
end

def MyApp.MySageTransactionTracer do
  use MyApp.TracerBuilder, [metrics?: true, trace_span: false, logging: false]
# ...

with_tracer(sage, MySageTransactionTracer)

The issue is the macro addition and the ceremony to always have to define a tracer module.

All that could be avoided with we dispatch an extra argument on with_tracer that could be accessed as last argument on handle_event.

What do you think? Would you be interested on such change?

Example

Sage.new()
|> Sage.run(:step1, &fun_one/2)
|> Sage.run_async(:step2, &fun_two/2, :noop)
|> Sage.run(:step3, &fun_three/2)
|> Sage.transaction(Repo, args)

Expected Behavior

When step3 returns {:error, reason}, operations in step1 and step3 should be rolled back and :noop for step2.

Current Behavior

The following error is thrown:

** (BadFunctionError) expected a function, got: :noop
    (sage) lib/sage/executor.ex:299: Sage.Executor.apply_compensation_fun/5
    (sage) lib/sage/executor.ex:271: Sage.Executor.safe_apply_compensation_fun/5
    (sage) lib/sage/executor.ex:264: Sage.Executor.execute_compensation/2
    (sage) lib/sage/executor.ex:240: Sage.Executor.execute_compensations/4
    (sage) lib/sage/executor.ex:29: Sage.Executor.execute/2
    (sage) lib/sage.ex:422: anonymous fn/3 in Sage.transaction/3
    (ecto_sql) lib/ecto/adapters/sql.ex:898: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1415: DBConnection.run_transaction/4
    (sage) lib/sage.ex:421: Sage.transaction/3

Why this is a problem?

This behavior will become an issue if you're composing multiple sages.

Hi! I'm implementing opentelemetry tracing on my job application and collecting the effects_so_far as tracing attributes would be a very handful to understand some Sage pipelines. Today Sage provides the hook's name, the action, and the shared opts.

What do you all think about this idea?

There is a logical flaw in current Sage version, imagine that we have a transaction that inserts something in the database and no compensation for it (because the effect would be rolled back on transaction abort). If there is a retry on one of preceding steps to it - we may insert multiple records and succeed.

Fortunately, it's pretty easy to implement. We need to support transaction execution adapter which works similarly to inspection adapter and is called each time we want to apply or compensate a step. The adapter needs to take care of wrapping the sage and executing it. Additionally, it should allow executing code after it's committed (for #23).

In future, we would need something like sage_ecto package (which can be built in with if Code.ensure_loaded?(Ecto) for now).

We can write helpers that simplify building a test case that would take a sage and test possible failures in each it's stage, to make sure operations and compensations are all working properly.

The error is occurring because the retry_opts() function, for retry_limit, is set only once for a specific function. However, this function has two different error cases for two different retry_limit approaches.

In one case, it should be retried 3 times, and in another case, it should be retried 10 times.

 test "full errors retries but ok in last effect" do

      socket_close_retries = GetPismoCardDetails.retry_info(:socket_close_retries) - 1
      server_error_retries = GetPismoCardDetails.retry_info(:server_error_retries) - 1

      for _i <- 1..server_error_retries do
        expect(PismoMock, :get_card_pci_information, fn _ ->
          {:error, {:server_error, %{"error" => "Internal Server Error"}}}
        end)
      end

      for _i <- 1..socket_close_retries do
        expect(PismoMock, :get_card_pci_information, fn _ ->
          {:error, "socket closed"}
        end)
      end

      expect(PismoMock, :get_card_pci_information, fn _ ->
        {:ok, %{pan: "#####"}}
      end)

      {:ok, "#####"} =
        GetPismoCardDetails.execute(%{
          pismo_card_id: :rand.uniform(10_000),
          pismo_account_id: :rand.uniform(10_000)
        })
    end
  end

  defp get_card_pci_information_compensation(
         {:server_error, %{"error" => "Internal Server Error"}} = _error,
         _effects_so_far,
         %{pismo_card_id: pismo_card_id}
       ) do
    Logger.warning("Failed to fetch pan due to Internal Server Error, retrying it",
      pismo_card_id: pismo_card_id
    )

    {:retry, retry_limit: 3}
  end

  defp get_card_pci_information_compensation(
         "socket closed" = error,
         _effects_so_far,
         %{pismo_card_id: pismo_card_id}
       ) do
    Logger.warning("Failed to fetch card pan due to #{inspect({:error, error})} retrying it",
      pismo_card_id: pismo_card_id
    )

    {:retry, retry_limit: 10}
  end

• Make sure all error messages are clean and have no spelling mistakes
• Make readme and module docs are clean and have no spelling mistakes
• Form a simple terminology (eg. use hook for finally) to make docs simpler to understand
• Write sample compensation adapter that retries failed compensations over time
• Create Sage.Operation protocol and refactor existing function operations to work on top of it
• Implement simple Retry policies to hook into Sage and let it be extendable
• Implement simple Cache policies to hook into Sage and let it be extendable
• Get more feedback from friends :)

As :source_ref in mix.exs is set to "v#\{@version\}", links to source in hexdocs lead to that kind of URLs: https://github.com/Nebo15/sage/blob/v#{@version}/lib/sage.ex#L229

Now that would be an easy pull request but I guess there is a reason for the escaping.

Hiya @AndrewDryga - we're using Sage quite a lot and have recently upgraded to elixir 1.10. We've noticed a lot of warnings on 0.4.0, and I'd forked with the intention of cleaning them up and pull requesting, but I see that they're all actually fixed on master. Keen to understand your appetite for a 0.5.0 or 1.0.0 release (appears there are some breaking API changes), and if there's anything I can do to help get there?

  lib/sage.ex:416                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                       
warning: System.stacktrace/0 outside of rescue/catch clauses is deprecated. If you want to support only Elixir v1.7+, you must access __STACKTRACE__ inside a rescue/catch. If you want to support earlier Elixir versions, move System.stacktrace/0 inside a rescue/ca
tch                                                                                                                                                                                                                                                                    
  lib/sage/executor.ex:90                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                       
warning: "else" shouldn't be used as the only clause in "defp", use "case" instead                                                                                                                                                                                     
  lib/sage/executor.ex:157                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: System.stacktrace/0 outside of rescue/catch clauses is deprecated. If you want to support only Elixir v1.7+, you must access __STACKTRACE__ inside a rescue/catch. If you want to support earlier Elixir versions, move System.stacktrace/0 inside a rescue/ca
tch                                                                                                                                                                                                                                                                    
  lib/sage/executor.ex:170                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: "else" shouldn't be used as the only clause in "defp", use "case" instead                                                                                                                                                                                     
  lib/sage/executor.ex:173                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: System.stacktrace/0 outside of rescue/catch clauses is deprecated. If you want to support only Elixir v1.7+, you must access __STACKTRACE__ inside a rescue/catch. If you want to support earlier Elixir versions, move System.stacktrace/0 inside a rescue/ca
tch                                                                                                                                                                                                                                                                    
  lib/sage/executor.ex:186                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: System.stacktrace/0 outside of rescue/catch clauses is deprecated. If you want to support only Elixir v1.7+, you must access __STACKTRACE__ inside a rescue/catch. If you want to support earlier Elixir versions, move System.stacktrace/0 inside a rescue/ca
tch                                                                                                                                                                                                                                                                    
  lib/sage/executor.ex:292                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: System.stacktrace/0 outside of rescue/catch clauses is deprecated. If you want to support only Elixir v1.7+, you must access __STACKTRACE__ inside a rescue/catch. If you want to support earlier Elixir versions, move System.stacktrace/0 inside a rescue/ca
tch                                                                                                                                                                                                                                                                    
  lib/sage/executor.ex:337                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                       
warning: Inspect.Algebra.surround_many/5 is deprecated. Use Inspect.Algebra.container_doc/6 instead                                                                                                                                                                    
  lib/sage/inspect.ex:13: Inspect.Sage.inspect/2```

We should educate developers to never send events from within the transaction.

A this is a very common case I've dealt with. Usually, we spawn a lot of events and it's a very bad practice to when code that spawns them is used within a database transaction - we don't have a way to cancel them but the transaction can be rolled back. But you never know all the details and you never know who would use your code within a transaction in one of the new business processes.

Sometimes sending events before a transaction is committed creates a race condition. When there is a code that listens for events and reads the value from the database it would crash, because the value can be not there yet.

Also, sometimes the high-level code just knows better what events it should send and when, so we don't want to fire them from within a context at all. A good example from practice - when a user is signed up with a pre-defined plan (by a voucher code), we don't want to broadcast subscription.created event right there, because it would come before the user.created event.

Speaking in terms of code, this is bad:

defmodule MyApp.Domain.Users.UseCase do
  def sign_up(attrs) do
    MyApp.Domain.Repo.transaction(fn ->
      # ... create an account ...
      MyApp.broadcast(MyApp.PubSub, "account.created", account: account)
      account
    end)
  end
end

Good:

defmodule MyApp.Domain.Users.UseCase do
  def sign_up(attrs) do
    with {:ok, {account, events}} <- create_account(attrs) do
      for {:event, kind, metadata} <- events do
        MyApp.broadcast(MyApp.PubSub, kind, metadata)
      end

      {:ok, account}
    end
  end

  defp create_account(attrs) do
    MyApp.Domain.Repo.transaction(fn ->
      # ... create an account ...
      {account, [{:event, "account.created", account: account}]}
    end)
  end
end

To handle those scenarios we can threat events as effects, return them as the third element of :ok tuple for compensations, accumulate (with scoping them by stage name to don't add a second place where developer should keep track on the names) and only fire when the transaction is succeeded from final callback.

Actionable items:

• Make sure events are sent only AFTER transaction is committed;
• Leverage persistence layer so that those events should be set at least once (if Sage fails between commit and events dispatching we need to retry later stage).

From the docs https://hexdocs.pm/sage/readme.html:

# Transaction behaviour:
# @callback transaction(attrs :: map()) :: {:ok, last_effect :: any(), all_effects :: map()} | {:error, reason :: any()}

But when you actually check the code, the success type is 2-tuple, not 3-tuple