From ed91bc8cafd0a1066c53eae321b0dd33d5d061e2 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Thu, 29 Oct 2026 19:54:56 +0000 Subject: [PATCH 1/2] deps: update dependency flagsmith/sdk-spec to v0.7.0 --- .github/workflows/sdk-spec.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/sdk-spec.yml b/.github/workflows/sdk-spec.yml index 103da1c..335aeac 100644 --- a/.github/workflows/sdk-spec.yml +++ b/.github/workflows/sdk-spec.yml @@ -11,7 +11,7 @@ permissions: jobs: sdk-spec: # Keep the ref in `uses:` and `spec-ref` in sync. - uses: Flagsmith/sdk-spec/.github/workflows/run-spec.yml@v0.3.0 + uses: Flagsmith/sdk-spec/.github/workflows/run-spec.yml@v0.7.0 with: - spec-ref: v0.3.0 + spec-ref: v0.7.0 language: elixir From 9caa72a5eee2f8af4bc6f41e044bee892917df45 Mon Sep 17 00:00:00 2001 From: "flagsmithsmith[bot]" <346993984+flagsmithsmith[bot]@users.noreply.github.com> Date: Thu, 29 Oct 2026 13:40:22 +0000 Subject: [PATCH 2/2] feat: Experiment flags, event and exposure tracking Implements SDK spec v0.6.0's `@events` capability: - `Flagsmith.Client.get_experiment_flag/4` gets an identity's flag and, when the identity is enrolled in a running experiment on an enabled feature, sends a `$flag_exposure` event valued with the variant served. - `Flagsmith.Client.track_event/3` and `track_exposure_event/3` send events to Flagsmith's events API. - Events are buffered per environment key by a supervised processor and sent in batches on an interval, when the buffer fills, and on shutdown; failed batches are kept and retried. - Flags from the identities API now carry their `variant` and `experiment` metadata. - New options: `enable_events`, `events_api_url` and `events_flush_interval_milliseconds`. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 8541003e17c123f09192f79bc8afbaee38a894e0) --- HEXDOCS.md | 21 + lib/configuration/flagsmith_configuration.ex | 32 +- lib/flagsmith_client.ex | 190 ++++++- lib/flagsmith_client_events_processor.ex | 192 +++++++ lib/flagsmith_supervisor.ex | 11 +- lib/schemas/experiment.ex | 21 + lib/schemas/features/feature_state.ex | 6 +- .../features/feature_state_metadata.ex | 22 + lib/schemas/flag.ex | 10 +- .../events_processor_supervisor.ex | 35 ++ sdk-spec/steps.exs | 56 ++- test/flagsmith_configuration_test.exs | 34 +- test/flagsmith_events_test.exs | 470 ++++++++++++++++++ 13 files changed, 1072 insertions(+), 28 deletions(-) create mode 100644 lib/flagsmith_client_events_processor.ex create mode 100644 lib/schemas/experiment.ex create mode 100644 lib/schemas/features/feature_state_metadata.ex create mode 100644 lib/supervisors/events_processor_supervisor.ex create mode 100644 test/flagsmith_events_test.exs diff --git a/HEXDOCS.md b/HEXDOCS.md index 5e14726..86bfbfa 100644 --- a/HEXDOCS.md +++ b/HEXDOCS.md @@ -148,6 +148,21 @@ false = Flagsmith.Client.is_feature_enabled("my-other-feature-b") When you enable analytics reporting, besides having to start the `Flagsmith.Supervisor` for it to work, you need to query for flags/features/features values through the exposed functions in `Flagsmith.Client.get_flag/2` `Flagsmith.Client.is_feature_enabled/2` and `Flagsmith.Client.get_feature_value/2`. This will update the tracking of the number of times they're accessed. +#### Experiments & Events + +To measure experiments, enable events and start the `Flagsmith.Supervisor`. Getting a flag with `Flagsmith.Client.get_experiment_flag/4` records which variant the identity was served, and `Flagsmith.Client.track_event/3` records what the identity did: + +```elixir +config = Flagsmith.Client.new(environment_key: "MY_SDK_KEY", enable_events: true) + +%Flagsmith.Schemas.Flag{value: "hello treatment"} = + Flagsmith.Client.get_experiment_flag(config, "banner", "user-a", [%{trait_key: "plan", trait_value: "premium"}]) + +:ok = Flagsmith.Client.track_event(config, "checkout_completed", identifier: "user-a", value: 42, metadata: %{currency: "GBP"}) +``` + +Events are sent in batches, every `events_flush_interval_milliseconds`. + #### Identity & Traits When using local evaluation with identity based requests the provided traits will not be updated on the Flagsmith platform, and to properly calculate the flags for an identity you need to provide any relevant trait as well. For instance, to get the `flags` for the identity `"user-a"` with a given trait: @@ -202,3 +217,9 @@ And obviously, if the identity has that trait and you do the regular API call wi - `retries` -> the number of times the http adapter is allowed to retry failed calls to the Flagsmith API before deeming the response failed. Keep in mind that with local_evaluation and analytics, even if the requests fail after whatever number of retries you set here, they will keep being retried as their time-cycle resets (for the poller whatever is the environment_refresh_interval_milliseconds, for the analytics they're batched, and a dump is tried every 1 minute) - `enable_analytics` -> track feature queries by reporting automatically to the Flagsmith API analytics endpoint queries and lookups to features. This only works when using the `Flagsmith.Client` module functions such as `is_feature_enabled`, `get_feature_value` and `get_flag`. + +- `enable_events` -> send experiment exposures and tracked events to the Flagsmith events API (defaults to `false`). + +- `events_api_url` -> the base url of the Flagsmith events API (defaults to: `"https://events.api.flagsmith.com/"`) + +- `events_flush_interval_milliseconds` -> the time to wait between sending batches of events (defaults to 10_000 milliseconds) diff --git a/lib/configuration/flagsmith_configuration.ex b/lib/configuration/flagsmith_configuration.ex index 727bc8a..415de6a 100644 --- a/lib/configuration/flagsmith_configuration.ex +++ b/lib/configuration/flagsmith_configuration.ex @@ -2,6 +2,7 @@ defmodule Flagsmith.Configuration do require Logger @default_url "https://edge.api.flagsmith.com/api/v1/" + @default_events_url "https://events.api.flagsmith.com/" @environment_header "X-Environment-Key" @api_paths %{ @@ -9,7 +10,8 @@ defmodule Flagsmith.Configuration do identities: "/identities/", traits: "/traits/", analytics: "/analytics/flags/", - environment: "/environment-document/" + environment: "/environment-document/", + events: "/v1/events" } @min_refresh_interval if(Mix.env() == :test, do: 1, else: 60_000) @@ -24,7 +26,10 @@ defmodule Flagsmith.Configuration do enable_local_evaluation: false, environment_refresh_interval_milliseconds: 60_000, retries: 0, - enable_analytics: false + enable_analytics: false, + enable_events: false, + events_api_url: @default_events_url, + events_flush_interval_milliseconds: 10_000 ] @default_keys [ @@ -35,7 +40,10 @@ defmodule Flagsmith.Configuration do :enable_local_evaluation, :environment_refresh_interval_milliseconds, :retries, - :enable_analytics + :enable_analytics, + :enable_events, + :events_api_url, + :events_flush_interval_milliseconds ] @type t() :: %__MODULE__{ @@ -47,12 +55,18 @@ defmodule Flagsmith.Configuration do enable_local_evaluation: boolean(), environment_refresh_interval_milliseconds: non_neg_integer(), retries: non_neg_integer(), - enable_analytics: boolean() + enable_analytics: boolean(), + enable_events: boolean(), + events_api_url: String.t(), + events_flush_interval_milliseconds: pos_integer() } @doc false def default_url(), do: @default_url + @doc false + def default_events_url(), do: @default_events_url + @doc false def environment_header(), do: @environment_header @@ -144,6 +158,16 @@ defmodule Flagsmith.Configuration do :enable_analytics -> is_boolean(value) || {:error, "needs to be true or false"} + + :enable_events -> + is_boolean(value) || {:error, "needs to be true or false"} + + :events_api_url -> + is_binary(value) || {:error, "needs to be a String.t()"} + + :events_flush_interval_milliseconds -> + (is_integer(value) and value > 0) || + {:error, "needs to be an integer bigger than 0"} end end diff --git a/lib/flagsmith_client.ex b/lib/flagsmith_client.ex index de44558..002edf3 100644 --- a/lib/flagsmith_client.ex +++ b/lib/flagsmith_client.ex @@ -1,22 +1,36 @@ defmodule Flagsmith.Client do + require Logger + alias Flagsmith.Schemas alias Flagsmith.Configuration + alias Flagsmith.Client.Events @api_paths Flagsmith.Configuration.api_paths() @environment_header Flagsmith.Configuration.environment_header() + @flag_exposure_event Events.Processor.flag_exposure_event() @type tesla_header_list :: [{String.t(), String.t()}] @type config_or_env :: Configuration.t() | Keyword.t() | Schemas.Environment.t() + @type event_value :: String.t() | number() | boolean() | nil + @type event_opts :: [ + identifier: String.t(), + value: event_value(), + traits: list(map() | Schemas.Traits.Trait.t()), + metadata: map() + ] @doc false - @spec user_agent() :: String.t() - def user_agent do + @spec version() :: String.t() + def version do # x-release-please-start-version - version = "2.3.1" + "2.3.1" # x-release-please-end - "flagsmith-elixir-sdk/#{version}" end + @doc false + @spec user_agent() :: String.t() + def user_agent, do: "flagsmith-elixir-sdk/#{version()}" + @doc """ Create a `t:Flagsmith.Configuration.t/0` struct with the desired settings to use in requests. @@ -203,6 +217,117 @@ defmodule Flagsmith.Client do end end + @doc """ + Returns a `t:Flagsmith.Schemas.Flag.t/0` by name, evaluated for an identity as + `get_identity_flags/4` does, and records which variant of the feature the identity + was exposed to. + + The exposure is sent as a `"$flag_exposure"` event, valued with the variant the + identity was served, only when the flag is enabled and the identity is enrolled in a + running experiment on the feature. + + Requires `:enable_events` to be `true` and the `Flagsmith.Supervisor` to be running. + Returns what `get_flag/2` returns for the identity's flags, or an `:error` tuple if + unable to get them. + """ + @spec get_experiment_flag( + Configuration.t() | Keyword.t(), + feature_name :: String.t(), + identifier :: String.t(), + list(map() | Schemas.Traits.Trait.t()) + ) :: Schemas.Flag.t() | :not_found | term() + def get_experiment_flag(configuration_or_opts, feature_name, identifier, traits \\ []) + + def get_experiment_flag(%Configuration{enable_events: false}, _, _, _), + do: {:error, :events_disabled} + + def get_experiment_flag(%Configuration{} = config, feature_name, identifier, traits) do + with {:ok, flags} <- get_identity_flags(config, identifier, traits) do + flag = get_flag(flags, feature_name) + + case exposure_skip_reason(flag) do + nil -> + track_exposure_event(config, feature_name, + identifier: identifier, + value: flag.variant, + traits: traits, + metadata: %{experiment_id: flag.experiment.id} + ) + + reason -> + Logger.debug("Not sending #{@flag_exposure_event}: #{reason}", + feature_name: feature_name + ) + end + + flag + end + end + + def get_experiment_flag(opts, feature_name, identifier, traits) when is_list(opts), + do: get_experiment_flag(new(opts), feature_name, identifier, traits) + + @doc """ + Sends an event to Flagsmith, e.g. a conversion for experiments to measure. + + Options: + - `:identifier` -> the identity the event happened to + - `:value` -> a string, number or boolean, sent as a string + - `:traits` -> the identity's traits, in the format `get_identity_flags/4` takes + - `:metadata` -> a map of additional data + + Events are buffered and sent in batches, so this returns `:ok` without waiting for + the event to be sent. Requires `:enable_events` to be `true` and the + `Flagsmith.Supervisor` to be running. Event names starting with `$` are reserved + for Flagsmith's own events. + """ + @spec track_event(Configuration.t() | Keyword.t(), event :: String.t(), event_opts()) :: + :ok | {:error, term()} + def track_event(configuration_or_opts, event, opts \\ []) + + def track_event(%Configuration{enable_events: false}, _, _), + do: {:error, :events_disabled} + + def track_event(%Configuration{}, "$" <> _ = event, _) when event != @flag_exposure_event, + do: {:error, {:reserved_event_name, event}} + + def track_event(%Configuration{} = config, event, opts) when is_binary(event), + do: buffer_event(config, event, nil, opts) + + def track_event(opts, event, event_opts) when is_list(opts), + do: track_event(new(opts), event, event_opts) + + @doc """ + Sends a `"$flag_exposure"` event to Flagsmith, recording that an identity was shown + a variant of a feature. `get_experiment_flag/4` sends these by itself. + + Takes the same options as `track_event/3`, with `:value` being the variant, and + `:identifier` being required to reconcile the exposure with the identity's events. + Exposures equal in feature, identifier and value are sent once per flush interval. + """ + @spec track_exposure_event( + Configuration.t() | Keyword.t(), + feature_name :: String.t(), + event_opts() + ) :: :ok | {:error, term()} + def track_exposure_event(configuration_or_opts, feature_name, opts \\ []) + + def track_exposure_event(%Configuration{enable_events: false}, _, _), + do: {:error, :events_disabled} + + def track_exposure_event(%Configuration{} = config, feature_name, opts) do + case Keyword.get(opts, :identifier) do + identifier when is_binary(identifier) and identifier != "" -> + buffer_event(config, @flag_exposure_event, feature_name, opts) + + _ -> + {:error, :identifier_required} + end + end + + def track_exposure_event(opts, feature_name, event_opts) when is_list(opts), + do: track_exposure_event(new(opts), feature_name, event_opts) + @doc """ Return all flags from an environment as a list. If a `t:Flagsmith.Schemas.Environment.t/0` is passed instead of a @@ -356,6 +481,24 @@ defmodule Flagsmith.Client do when is_list(opts) and is_map(tracking) and not is_struct(tracking), do: analytics_track(new(opts), tracking) + @doc false + # Submits a batch of events to the Flagsmith events API. + @spec events_track(Configuration.t(), list(Events.Processor.event())) :: + {:ok, term()} | {:error, term()} + def events_track(%Configuration{events_api_url: events_api_url} = config, events) do + case Tesla.post( + http_client(%{config | api_url: events_api_url}), + @api_paths.events, + %{events: events} + ) do + {:ok, %{status: status, body: body}} when status >= 200 and status < 300 -> + {:ok, body} + + error_resp -> + return_error(error_resp) + end + end + @doc false # Given an `t:Flagsmith.Schemas.Environment.t/0` or a list composed of # `t:Flagsmith.Schemas.Environment.FeatureState.t/0` or @@ -387,6 +530,45 @@ defmodule Flagsmith.Client do feature_flag end + defp exposure_skip_reason(%Schemas.Flag{enabled: false}), do: "feature is disabled" + + defp exposure_skip_reason(%Schemas.Flag{experiment: %{in_experiment: true}, variant: nil}), + do: "flag has no variant" + + defp exposure_skip_reason(%Schemas.Flag{experiment: %{in_experiment: true}}), do: nil + + defp exposure_skip_reason(%Schemas.Flag{}), do: "identity is not in a running experiment" + defp exposure_skip_reason(_), do: "feature not found" + + defp buffer_event(%Configuration{} = config, event, feature_name, opts) do + with {:ok, value} <- event_value(Keyword.get(opts, :value)) do + Events.Processor.track( + %{ + event: event, + feature_name: feature_name, + identifier: Keyword.get(opts, :identifier), + value: value, + traits: event_traits(Keyword.get(opts, :traits)), + metadata: Map.put(Keyword.get(opts, :metadata) || %{}, :sdk_version, version()), + timestamp: System.os_time(:millisecond) + }, + config + ) + end + end + + defp event_value(value) when is_nil(value) or is_binary(value), do: {:ok, value} + defp event_value(value) when is_number(value) or is_boolean(value), do: {:ok, to_string(value)} + defp event_value(value), do: {:error, {:invalid_value, value}} + + defp event_traits([_ | _] = traits) do + traits + |> Schemas.Traits.Trait.from() + |> Map.new(fn %{trait_key: key, trait_value: value} -> {key, value} end) + end + + defp event_traits(_), do: nil + defp build_identity_params(identifier, [_ | _] = traits, transient) do %{ identifier: identifier, diff --git a/lib/flagsmith_client_events_processor.ex b/lib/flagsmith_client_events_processor.ex new file mode 100644 index 0000000..b2c8bfa --- /dev/null +++ b/lib/flagsmith_client_events_processor.ex @@ -0,0 +1,192 @@ +defmodule Flagsmith.Client.Events.Processor do + use GenServer + require Logger + @moduledoc false + + alias Flagsmith.Configuration + + @type processor_identifier :: {__MODULE__, String.t()} + @type environment_key :: String.t() + @type event :: %{ + required(:event) => String.t(), + required(:feature_name) => String.t() | nil, + required(:identifier) => String.t() | nil, + required(:value) => String.t() | nil, + required(:traits) => map() | nil, + required(:metadata) => map(), + required(:timestamp) => integer() + } + + @flag_exposure_event "$flag_exposure" + @max_buffered_events 1000 + + @enforce_keys [:configuration] + defstruct [ + :configuration, + buffer: [], + exposure_keys: MapSet.new() + ] + + ################################# + ########### API + ################################# + + @doc false + @spec flag_exposure_event() :: String.t() + def flag_exposure_event(), do: @flag_exposure_event + + @doc """ + Buffers an event to be sent to the Flagsmith events API on the next flush, starting + a processor for the configuration's environment key in case one is not running. + + Exposures equal in feature, identifier and value to one already buffered are + dropped, as they add no information. + + It's a non-blocking operation. + """ + @spec track(event(), Configuration.t()) :: :ok | {:error, term()} + def track(event, %Configuration{environment_key: env_key} = config) do + case whereis(env_key) do + pid when is_pid(pid) -> + GenServer.cast(pid, {:track, event}) + + :undefined -> + case __MODULE__.Supervisor.start_child(config) do + {:ok, pid} -> GenServer.cast(pid, {:track, event}) + {:error, {:already_started, pid}} -> GenServer.cast(pid, {:track, event}) + error -> error + end + end + end + + @doc """ + Sends every buffered event to the Flagsmith events API, returning once done. + """ + @spec flush(environment_key()) :: :ok | :noop | {:error, term()} + def flush(environment_key) do + case whereis(environment_key) do + pid when is_pid(pid) -> GenServer.call(pid, :flush) + :undefined -> :noop + end + end + + @doc false + @spec processor_id(environment_key()) :: processor_identifier() + def processor_id(environment_key), do: {__MODULE__, environment_key} + + @doc false + @spec via_tuple(environment_key()) :: + {:via, Registry, {Flagsmith.Registry, processor_identifier()}} + def via_tuple(environment_key), + do: {:via, Registry, {Flagsmith.Registry, processor_id(environment_key)}} + + @doc """ + Returns the pid of the Events.Processor process related to a given environment key, + or :undefined if one can't be found. + """ + @spec whereis(environment_key()) :: :undefined | pid + def whereis(environment_key) do + case Registry.lookup(Flagsmith.Registry, processor_id(environment_key)) do + [] -> :undefined + [{pid, _}] -> pid + end + end + + @doc """ + Starts and links a gen_server represented by this module, using a + `t:Flagsmith.Configuration.t/0` as the basis to derive its registration name and + inner details. + """ + @spec start_link(Configuration.t()) :: GenServer.on_start() + def start_link(%Configuration{environment_key: environment_key} = config), + do: GenServer.start_link(__MODULE__, config, name: via_tuple(environment_key)) + + ################################# + ########### GenServer Implementation / Internal + ################################# + + @impl GenServer + def init(%Configuration{} = config) do + Process.flag(:trap_exit, true) + schedule_flush(config) + {:ok, %__MODULE__{configuration: config}} + end + + @impl GenServer + def handle_cast({:track, event}, %__MODULE__{} = data) do + case exposure_key(event) do + nil -> + {:noreply, buffer(data, event)} + + key -> + case MapSet.member?(data.exposure_keys, key) do + true -> + Logger.debug( + "Skipping duplicate #{@flag_exposure_event} event", + feature_name: event.feature_name, + identifier: event.identifier + ) + + {:noreply, data} + + false -> + {:noreply, + buffer(%{data | exposure_keys: MapSet.put(data.exposure_keys, key)}, event)} + end + end + end + + @impl GenServer + def handle_call(:flush, _from, %__MODULE__{} = data) do + {result, data} = do_flush(data) + {:reply, result, data} + end + + @impl GenServer + def handle_info(:flush, %__MODULE__{configuration: config} = data) do + {_, data} = do_flush(data) + schedule_flush(config) + {:noreply, data} + end + + def handle_info(_, data), do: {:noreply, data} + + @impl GenServer + def terminate(_reason, %__MODULE__{} = data), do: do_flush(data) + + defp exposure_key(%{event: @flag_exposure_event} = event), + do: {event.feature_name, event.identifier, event.value} + + defp exposure_key(_), do: nil + + defp buffer(%__MODULE__{buffer: buffer} = data, event) do + data = %{data | buffer: [event | buffer]} + + case length(data.buffer) >= @max_buffered_events do + true -> elem(do_flush(data), 1) + false -> data + end + end + + defp do_flush(%__MODULE__{buffer: []} = data), do: {:noop, data} + + defp do_flush(%__MODULE__{configuration: config, buffer: buffer} = data) do + events = Enum.reverse(buffer) + + case Flagsmith.Client.events_track(config, events) do + {:ok, _} -> + {:ok, %{data | buffer: [], exposure_keys: MapSet.new()}} + + {:error, reason} = error -> + Logger.warning("Failed to send events to Flagsmith, keeping them to retry", + reason: inspect(reason), + events: length(events) + ) + + {error, %{data | buffer: Enum.take(buffer, @max_buffered_events)}} + end + end + + defp schedule_flush(%Configuration{events_flush_interval_milliseconds: interval}), + do: Process.send_after(self(), :flush, interval) +end diff --git a/lib/flagsmith_supervisor.ex b/lib/flagsmith_supervisor.ex index 81249c2..ec74b62 100644 --- a/lib/flagsmith_supervisor.ex +++ b/lib/flagsmith_supervisor.ex @@ -31,13 +31,13 @@ defmodule Flagsmith.Supervisor do `{:ok, pid} = Flagsmith.Supervisor.start_link()` - This supervisor starts 3 additional processes, a local `Registry` and two dynamic - supervisors responsible for starting Pollers and Analytics processes (usually + This supervisor starts 4 additional processes, a local `Registry` and three dynamic + supervisors responsible for starting Pollers, Analytics and Events processes (usually a single one, but since they are isolated by sdk key used, if you use multiple sdks to access different environments in Flagsmith, in the same application you'll - end up with that same amount of processes, number of environments times 2). - The pollers and analytics are only started once there are interactions with either, - and they stay until application shutdown. + end up with that same amount of processes, number of environments times 3). + The pollers, analytics and events processes are only started once there are + interactions with them, and they stay until application shutdown. """ def child_spec(_), @@ -57,6 +57,7 @@ defmodule Flagsmith.Supervisor do children = [ {Registry, [keys: :unique, name: Flagsmith.Registry]}, Flagsmith.Client.Analytics.Processor.Supervisor, + Flagsmith.Client.Events.Processor.Supervisor, Flagsmith.Client.Poller.Supervisor ] diff --git a/lib/schemas/experiment.ex b/lib/schemas/experiment.ex new file mode 100644 index 0000000..19e328b --- /dev/null +++ b/lib/schemas/experiment.ex @@ -0,0 +1,21 @@ +defmodule Flagsmith.Schemas.Experiment do + use TypedEctoSchema + import Ecto.Changeset + + @moduledoc """ + Ecto schema representing the running Flagsmith experiment a flag was evaluated + under, and whether the identity is enrolled in it. + """ + + @primary_key false + typed_embedded_schema do + field(:id, :integer) + field(:name, :string) + field(:in_experiment, :boolean, default: false) + end + + @doc false + @spec changeset(__MODULE__.t(), map()) :: Ecto.Changeset.t() + def changeset(struct, params), + do: cast(struct, params, [:id, :name, :in_experiment]) +end diff --git a/lib/schemas/features/feature_state.ex b/lib/schemas/features/feature_state.ex index 75a0435..c347be8 100644 --- a/lib/schemas/features/feature_state.ex +++ b/lib/schemas/features/feature_state.ex @@ -18,8 +18,10 @@ defmodule Flagsmith.Schemas.Features.FeatureState do field(:feature_state_value, Types.AnyOf, types: [:string, :integer, :float, :boolean]) field(:id, :integer) field(:identity, :integer) + field(:variant, :string) embeds_one(:feature, Flagsmith.Schemas.Features.Feature) + embeds_one(:metadata, __MODULE__.Metadata) end @doc false @@ -33,9 +35,11 @@ defmodule Flagsmith.Schemas.Features.FeatureState do :feature_state_value, :feature_segment, :environment, - :enabled + :enabled, + :variant ]) |> cast_embed(:feature, required: true) + |> cast_embed(:metadata) end @doc false diff --git a/lib/schemas/features/feature_state_metadata.ex b/lib/schemas/features/feature_state_metadata.ex new file mode 100644 index 0000000..7593244 --- /dev/null +++ b/lib/schemas/features/feature_state_metadata.ex @@ -0,0 +1,22 @@ +defmodule Flagsmith.Schemas.Features.FeatureState.Metadata do + use TypedEctoSchema + import Ecto.Changeset + + @moduledoc """ + Ecto schema representing the additional, non-evaluation data Flagsmith returns + with a feature state. + """ + + @primary_key false + typed_embedded_schema do + embeds_one(:experiment, Flagsmith.Schemas.Experiment) + end + + @doc false + @spec changeset(__MODULE__.t(), map()) :: Ecto.Changeset.t() + def changeset(struct, params) do + struct + |> cast(params, []) + |> cast_embed(:experiment) + end +end diff --git a/lib/schemas/flag.ex b/lib/schemas/flag.ex index fa4824e..60cc3e1 100644 --- a/lib/schemas/flag.ex +++ b/lib/schemas/flag.ex @@ -1,7 +1,7 @@ defmodule Flagsmith.Schemas.Flag do use TypedEctoSchema - alias Flagsmith.Schemas.{Environment, Features} + alias Flagsmith.Schemas.{Environment, Experiment, Features} @moduledoc """ Ecto schema representing a Flagsmith Flag definition. @@ -12,6 +12,8 @@ defmodule Flagsmith.Schemas.Flag do field(:enabled, :boolean) field(:feature_name, :string) field(:value, :string) + field(:variant, :string) + embeds_one(:experiment, Experiment) end @doc false @@ -35,6 +37,8 @@ defmodule Flagsmith.Schemas.Flag do def from(%Features.FeatureState{ enabled: enabled, feature_state_value: value, + variant: variant, + metadata: metadata, feature: %Features.Feature{ name: name, id: id @@ -44,7 +48,9 @@ defmodule Flagsmith.Schemas.Flag do enabled: enabled, feature_name: name, value: value, - feature_id: id + feature_id: id, + variant: variant, + experiment: metadata && metadata.experiment } end end diff --git a/lib/supervisors/events_processor_supervisor.ex b/lib/supervisors/events_processor_supervisor.ex new file mode 100644 index 0000000..c0d83f1 --- /dev/null +++ b/lib/supervisors/events_processor_supervisor.ex @@ -0,0 +1,35 @@ +defmodule Flagsmith.Client.Events.Processor.Supervisor do + use DynamicSupervisor + + @moduledoc false + + require Logger + + def child_spec(_), + do: %{ + id: __MODULE__, + start: {__MODULE__, :start_link, []}, + type: :supervisor + } + + def start_link() do + Logger.info("::: ::: Starting #{__MODULE__} ::: :::") + DynamicSupervisor.start_link(__MODULE__, nil, name: __MODULE__) + end + + @spec start_child(Flagsmith.Configuration.t()) :: {:ok, pid()} | {:error, any()} + def start_child(%Flagsmith.Configuration{} = config) do + DynamicSupervisor.start_child( + __MODULE__, + %{ + id: Flagsmith.Client.Events.Processor, + start: {Flagsmith.Client.Events.Processor, :start_link, [config]}, + shutdown: 25_000, + restart: :permanent, + type: :worker + } + ) + end + + def init(_), do: DynamicSupervisor.init(strategy: :one_for_one) +end diff --git a/sdk-spec/steps.exs b/sdk-spec/steps.exs index caf6843..64ddba7 100644 --- a/sdk-spec/steps.exs +++ b/sdk-spec/steps.exs @@ -4,18 +4,37 @@ defmodule SDKSpec.Steps do alias Flagsmith.Client + alias Flagsmith.Schemas - defp as_spec_flags(flags) do - for flag <- Client.all_flags(flags) do - %{feature: flag.feature_name, enabled: flag.enabled, value: flag.value} - end - end + defp as_spec_flag(flag), + do: %{feature: flag.feature_name, enabled: flag.enabled, value: flag.value} + + defp as_spec_flags(flags), do: Enum.map(Client.all_flags(flags), &as_spec_flag/1) + + defp as_traits(traits), + do: for({key, value} <- traits, do: %{trait_key: key, trait_value: value}) # Create the SDK client def create_client(environment_key, api_url) do Client.new(environment_key: environment_key, api_url: api_url) end + # Create the SDK client with events enabled + def create_client_with_events(environment_key, api_url, events_api_url) do + # Events are sent by processes the Flagsmith supervisor runs. + case Flagsmith.Supervisor.start_link() do + {:ok, _} -> :ok + {:error, {:already_started, _}} -> :ok + end + + Client.new( + environment_key: environment_key, + api_url: api_url, + enable_events: true, + events_api_url: events_api_url + ) + end + # When the application gets the environment flags def get_environment_flags(client) do with {:ok, flags} <- Client.get_environment_flags(client) do @@ -25,10 +44,31 @@ defmodule SDKSpec.Steps do # When the application gets the flags for identity "{identifier}" with traits: def get_identity_flags(client, identifier, traits) do - traits = for {key, value} <- traits, do: %{trait_key: key, trait_value: value} - - with {:ok, flags} <- Client.get_identity_flags(client, identifier, traits) do + with {:ok, flags} <- Client.get_identity_flags(client, identifier, as_traits(traits)) do {:ok, as_spec_flags(flags)} end end + + # When the application gets the experiment flag "{feature}" for identity "{identifier}" with traits: + def get_experiment_flag(client, feature, identifier, traits) do + case Client.get_experiment_flag(client, feature, identifier, as_traits(traits)) do + %Schemas.Flag{} = flag -> {:ok, [as_spec_flag(flag)]} + :not_found -> {:ok, []} + error -> error + end + end + + # When the application tracks event "{event}" for identity "{identifier}" with value {value} and metadata: + def track_event(client, event, identifier, value, metadata) do + Client.track_event(client, event, identifier: identifier, value: value, metadata: metadata) + end + + # When the application tracks an exposure of feature "{feature}" for identity "{identifier}" with value {value} and metadata: + def track_exposure_event(client, feature, identifier, value, metadata) do + Client.track_exposure_event(client, feature, + identifier: identifier, + value: value, + metadata: metadata + ) + end end diff --git a/test/flagsmith_configuration_test.exs b/test/flagsmith_configuration_test.exs index ae3d8e1..5ccd110 100644 --- a/test/flagsmith_configuration_test.exs +++ b/test/flagsmith_configuration_test.exs @@ -9,13 +9,15 @@ defmodule Flagsmith.Configuration.Test do # If we used those from the configuration as we do in other tests, then # we wouldn't be asserting much @default_url "https://edge.api.flagsmith.com/api/v1/" + @default_events_url "https://events.api.flagsmith.com/" @environment_header "X-Environment-Key" @paths %{ flags: "/flags/", identities: "/identities/", traits: "/traits/", analytics: "/analytics/flags/", - environment: "/environment-document/" + environment: "/environment-document/", + events: "/v1/events" } alias Flagsmith.Configuration @@ -46,7 +48,10 @@ defmodule Flagsmith.Configuration.Test do enable_local_evaluation: false, environment_refresh_interval_milliseconds: 60_000, retries: 0, - enable_analytics: false + enable_analytics: false, + enable_events: false, + events_api_url: @default_events_url, + events_flush_interval_milliseconds: 10_000 } = Configuration.build(environment_key: "A") assert is_nil(flag_handler) @@ -54,6 +59,7 @@ defmodule Flagsmith.Configuration.Test do test "helper funs" do assert @default_url = Configuration.default_url() + assert @default_events_url = Configuration.default_events_url() assert @environment_header = Configuration.environment_header() assert @paths = Configuration.api_paths() @@ -73,7 +79,10 @@ defmodule Flagsmith.Configuration.Test do enable_local_evaluation: true, environment_refresh_interval_milliseconds: 50_000, retries: 5, - enable_analytics: true + enable_analytics: true, + enable_events: true, + events_api_url: "https://events.example.com/", + events_flush_interval_milliseconds: 1_000 ) assert %Configuration{ @@ -85,7 +94,10 @@ defmodule Flagsmith.Configuration.Test do enable_local_evaluation: true, environment_refresh_interval_milliseconds: 50_000, retries: 5, - enable_analytics: true + enable_analytics: true, + enable_events: true, + events_api_url: "https://events.example.com/", + events_flush_interval_milliseconds: 1_000 } = Configuration.build(environment_key: "B") assert is_function(default_handler, 1) @@ -138,6 +150,20 @@ defmodule Flagsmith.Configuration.Test do assert_raise RuntimeError, ~r/^enable_analytics needs to be/, fn -> Configuration.build(environment_key: "A", enable_analytics: "false") end + + assert_raise RuntimeError, ~r/^enable_events needs to be/, fn -> + Configuration.build(environment_key: "A", enable_events: "false") + end + + assert_raise RuntimeError, ~r/^events_api_url needs to be/, fn -> + Configuration.build(environment_key: "A", events_api_url: 0) + end + + Enum.each([0, "oi"], fn val -> + assert_raise RuntimeError, ~r/^events_flush_interval_milliseconds needs to be/, fn -> + Configuration.build(environment_key: "A", events_flush_interval_milliseconds: val) + end + end) end test "logs warning if unknown opt is passed" do diff --git a/test/flagsmith_events_test.exs b/test/flagsmith_events_test.exs new file mode 100644 index 0000000..0dd6452 --- /dev/null +++ b/test/flagsmith_events_test.exs @@ -0,0 +1,470 @@ +defmodule Flagsmith.Client.Events.Test do + use ExUnit.Case, async: false + + import ExUnit.CaptureLog + import Mox, only: [verify_on_exit!: 1, set_mox_global: 1, expect: 3, expect: 4] + + alias Flagsmith.Client + alias Flagsmith.Client.Events.Processor + alias Flagsmith.Schemas + + @environment_header Flagsmith.Configuration.environment_header() + @api_url Flagsmith.Configuration.default_url() + @api_paths Flagsmith.Configuration.api_paths() + @events_url "https://events.example.com/" + + setup :set_mox_global + setup :verify_on_exit! + + setup do + start_supervised!(Flagsmith.Supervisor) + + [ + sdk_version: Client.version(), + config: + Client.new( + environment_key: "events_test_key", + enable_events: true, + events_api_url: @events_url + ) + ] + end + + setup do + test_pid = self() + + [ + receive_events: fn tesla_env, _options -> + send(test_pid, {:events_request, tesla_env, Jason.decode!(tesla_env.body)}) + {:ok, %Tesla.Env{status: 202, body: ""}} + end + ] + end + + describe "get_experiment_flag" do + test "an identity in a running experiment is exposed to the variant it was served", %{ + config: config, + sdk_version: sdk_version, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, fn tesla_env, _options -> + assert tesla_env.url == Path.join([@api_url, @api_paths.identities]) <> "/" + + {:ok, + %Tesla.Env{ + status: 200, + body: %{ + "identifier" => "user_123", + "traits" => [], + "flags" => [ + %{ + "feature" => %{"id" => 1, "name" => "banner", "type" => "STANDARD"}, + "enabled" => true, + "feature_state_value" => "hello treatment", + "variant" => "treatment", + "metadata" => %{ + "experiment" => %{ + "id" => 7, + "name" => "banner experiment", + "in_experiment" => true + } + } + } + ] + } + }} + end) + + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + flag = + Client.get_experiment_flag(config, "banner", "user_123", [ + %{trait_key: "plan", trait_value: "premium"} + ]) + + :ok = Processor.flush(config.environment_key) + + # Then + assert %Schemas.Flag{ + feature_name: "banner", + enabled: true, + value: "hello treatment", + variant: "treatment", + experiment: %Schemas.Experiment{ + id: 7, + name: "banner experiment", + in_experiment: true + } + } = flag + + assert_receive {:events_request, tesla_env, %{"events" => [event]}} + assert tesla_env.method == :post + assert tesla_env.url == @events_url <> "v1/events" + assert {@environment_header, config.environment_key} in tesla_env.headers + + assert %{ + "event" => "$flag_exposure", + "feature_name" => "banner", + "identifier" => "user_123", + "value" => "treatment", + "traits" => %{"plan" => "premium"}, + "metadata" => %{"experiment_id" => 7, "sdk_version" => ^sdk_version}, + "timestamp" => timestamp + } = event + + assert is_integer(timestamp) + end + + for {case_name, flag_json, reason} <- [ + {"an identity outside a running experiment", + %{"enabled" => true, "feature_state_value" => "hello"}, + "identity is not in a running experiment"}, + {"an identity not enrolled in a running experiment", + %{ + "enabled" => true, + "feature_state_value" => "hello", + "metadata" => %{ + "experiment" => %{"id" => 7, "name" => "banner", "in_experiment" => false} + } + }, "identity is not in a running experiment"}, + {"an identity served no variant in a running experiment", + %{ + "enabled" => true, + "feature_state_value" => "hello", + "metadata" => %{ + "experiment" => %{"id" => 7, "name" => "banner", "in_experiment" => true} + } + }, "flag has no variant"}, + {"an identity in a running experiment on a disabled feature", + %{ + "enabled" => false, + "feature_state_value" => "hello", + "variant" => "treatment", + "metadata" => %{ + "experiment" => %{"id" => 7, "name" => "banner", "in_experiment" => true} + } + }, "feature is disabled"} + ] do + test "#{case_name} is not exposed", %{config: config} do + # Given + flag_json = + Map.put(unquote(Macro.escape(flag_json)), "feature", %{ + "id" => 1, + "name" => "banner", + "type" => "STANDARD" + }) + + expect(Tesla.Adapter.Mock, :call, fn _tesla_env, _options -> + {:ok, + %Tesla.Env{ + status: 200, + body: %{"identifier" => "user_123", "traits" => [], "flags" => [flag_json]} + }} + end) + + # When + log = + capture_log([level: :debug], fn -> + assert %Schemas.Flag{feature_name: "banner", value: "hello"} = + Client.get_experiment_flag(config, "banner", "user_123") + end) + + # Then + assert log =~ "Not sending $flag_exposure: #{unquote(reason)}" + assert :undefined = Processor.whereis(config.environment_key) + end + end + + test "an identity is not exposed to a feature that does not exist", %{config: config} do + # Given + expect(Tesla.Adapter.Mock, :call, fn _tesla_env, _options -> + {:ok, + %Tesla.Env{ + status: 200, + body: %{"identifier" => "user_123", "traits" => [], "flags" => []} + }} + end) + + # When + log = + capture_log([level: :debug], fn -> + assert :not_found = Client.get_experiment_flag(config, "banner", "user_123") + end) + + # Then + assert log =~ "Not sending $flag_exposure: feature not found" + assert :undefined = Processor.whereis(config.environment_key) + end + + test "an experiment flag Flagsmith fails to evaluate is an error" do + # Given + expect(Tesla.Adapter.Mock, :call, fn _tesla_env, _options -> + {:ok, %Tesla.Env{status: 500, body: "unavailable"}} + end) + + # When / Then + assert {:error, "unavailable"} = + Client.get_experiment_flag( + [environment_key: "events_test_key", enable_events: true], + "banner", + "user_123" + ) + end + + test "experiment flags require events to be enabled" do + assert {:error, :events_disabled} = + Client.get_experiment_flag([environment_key: "A"], "banner", "user_123") + end + end + + describe "track_event" do + test "a tracked event reaches Flagsmith with its value, traits and metadata", %{ + config: config, + sdk_version: sdk_version, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + assert :ok = + Client.track_event(config, "checkout_completed", + identifier: "user_123", + value: 42, + traits: [%{trait_key: "plan", trait_value: "premium"}], + metadata: %{"currency" => "GBP"} + ) + + :ok = Processor.flush(config.environment_key) + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => [event]}} + + assert %{ + "event" => "checkout_completed", + "feature_name" => nil, + "identifier" => "user_123", + "value" => "42", + "traits" => %{"plan" => "premium"}, + "metadata" => %{"currency" => "GBP", "sdk_version" => ^sdk_version} + } = event + end + + test "an event without value, traits or metadata reaches Flagsmith", %{ + sdk_version: sdk_version, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + assert :ok = + Client.track_event( + [environment_key: "events_test_key", enable_events: true], + "signed_up" + ) + + :ok = Processor.flush("events_test_key") + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => [event]}} + + assert %{ + "event" => "signed_up", + "identifier" => nil, + "value" => nil, + "traits" => nil, + "metadata" => %{"sdk_version" => ^sdk_version} + } = event + end + + test "event names reserved for Flagsmith are refused", %{config: config} do + assert {:error, {:reserved_event_name, "$checkout"}} = + Client.track_event(config, "$checkout", identifier: "user_123") + + assert :undefined = Processor.whereis(config.environment_key) + end + + test "event values other than strings, numbers and booleans are refused", %{config: config} do + assert {:error, {:invalid_value, %{"amount" => 42}}} = + Client.track_event(config, "checkout_completed", value: %{"amount" => 42}) + + assert :undefined = Processor.whereis(config.environment_key) + end + + test "tracking events requires events to be enabled" do + assert {:error, :events_disabled} = + Client.track_event([environment_key: "A"], "checkout_completed") + end + end + + describe "track_exposure_event" do + test "an exposure reaches Flagsmith as a $flag_exposure event", %{ + sdk_version: sdk_version, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + assert :ok = + Client.track_exposure_event( + [environment_key: "events_test_key", enable_events: true], + "banner", + identifier: "user_123", + value: true, + metadata: %{"experiment_id" => 1} + ) + + :ok = Processor.flush("events_test_key") + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => [event]}} + + assert %{ + "event" => "$flag_exposure", + "feature_name" => "banner", + "identifier" => "user_123", + "value" => "true", + "metadata" => %{"experiment_id" => 1, "sdk_version" => ^sdk_version} + } = event + end + + test "an identity exposed to the same variant twice before a flush is sent once", %{ + config: config, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + log = + capture_log([level: :debug], fn -> + for value <- ["control", "control", "treatment"] do + :ok = + Client.track_exposure_event(config, "banner", identifier: "user_123", value: value) + end + + :ok = Processor.flush(config.environment_key) + end) + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => events}} + assert ["control", "treatment"] = Enum.map(events, & &1["value"]) + assert log =~ "Skipping duplicate $flag_exposure event" + end + + test "an exposure without an identifier is refused", %{config: config} do + assert {:error, :identifier_required} = + Client.track_exposure_event(config, "banner", value: "control") + + assert {:error, :identifier_required} = + Client.track_exposure_event(config, "banner", identifier: "", value: "control") + + assert :undefined = Processor.whereis(config.environment_key) + end + + test "tracking exposures requires events to be enabled" do + assert {:error, :events_disabled} = + Client.track_exposure_event([environment_key: "A"], "banner", + identifier: "user_123" + ) + end + end + + describe "sending events" do + test "buffered events are sent on every flush interval", %{receive_events: receive_events} do + # Given + config = + Client.new( + environment_key: "events_test_key", + enable_events: true, + events_api_url: @events_url, + events_flush_interval_milliseconds: 10 + ) + + expect(Tesla.Adapter.Mock, :call, 2, receive_events) + + # When + :ok = Client.track_event(config, "first") + assert_receive {:events_request, _, %{"events" => [%{"event" => "first"}]}}, 500 + :ok = Client.track_event(config, "second") + + # Then + assert_receive {:events_request, _, %{"events" => [%{"event" => "second"}]}}, 500 + end + + test "events Flagsmith fails to receive are sent again on the next flush", %{ + config: config, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, fn _tesla_env, _options -> + {:ok, %Tesla.Env{status: 503, body: "unavailable"}} + end) + + expect(Tesla.Adapter.Mock, :call, receive_events) + :ok = Client.track_event(config, "checkout_completed", identifier: "user_123") + + # When + log = + capture_log(fn -> + assert {:error, "unavailable"} = Processor.flush(config.environment_key) + end) + + :ok = Client.track_event(config, "signed_out", identifier: "user_123") + :ok = Processor.flush(config.environment_key) + + # Then + assert log =~ "Failed to send events to Flagsmith, keeping them to retry" + + assert_receive {:events_request, _tesla_env, %{"events" => events}} + assert ["checkout_completed", "signed_out"] = Enum.map(events, & &1["event"]) + end + + test "a full buffer of events is sent without waiting for the flush interval", %{ + config: config, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + + # When + for n <- 1..1000, do: :ok = Client.track_event(config, "event_#{n}") + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => events}}, 1000 + assert length(events) == 1000 + assert :noop = Processor.flush(config.environment_key) + end + + test "buffered events are sent when the application stops", %{ + config: config, + receive_events: receive_events + } do + # Given + expect(Tesla.Adapter.Mock, :call, receive_events) + :ok = Client.track_event(config, "checkout_completed") + + # When + :ok = stop_supervised(Flagsmith.Supervisor) + + # Then + assert_receive {:events_request, _tesla_env, %{"events" => [event]}} + assert %{"event" => "checkout_completed"} = event + end + + test "nothing is sent when there are no events", %{config: config} do + # Given + expect(Tesla.Adapter.Mock, :call, fn _, _ -> {:ok, %Tesla.Env{status: 202}} end) + :ok = Client.track_event(config, "checkout_completed") + :ok = Processor.flush(config.environment_key) + + # When / Then + assert :noop = Processor.flush(config.environment_key) + assert :noop = Processor.flush("unused_environment_key") + end + end +end