Consent feature

CHANGELOG.md

@@ -1,4 +1,8 @@
 ## XX.XX.XX
+- ! Minor breaking change ! The "Consent required" option of the SDK Behavior Settings now applies. While it is on, the SDK collects nothing until the app gives consent; while it is off, the SDK works as if every consent were given, also in apps that require consent.
+
+- Added user consent. With `setRequiresConsent(true)` a feature works only after its consent is given, at start with `setConsentEnabled` or later with `giveConsent`, `giveConsentAll` or `setConsent`, and stops when the consent is removed with `removeConsent`, `removeConsentAll` or `setConsent`. The SDK sends every change to the server. The feature names are in `cly::consent`, and `cly::consent::allFeatures()` lists them all. `createFeatureGroup` and `setConsentFeatureGroup` handle groups of features; `getConsent` and `checkAllConsent` show the current state.
+- While consent is required, changing the device ID without merge removes all consent, so the app gives it again for the new user.
 - Added the health check feature. The SDK now sends a health check request once after `start()`. Like any request, it updates the user's last seen time on the server.
 - Added the SDK name and version (`sdk_name` and `sdk_version`) to every request, including the remote config and SDK Behavior Settings fetches. Before, only the begin session request carried them.
 - Added `multiplyCustomUserDetail` for multiplying the numeric value of a custom user property on the server.
@@ -7,6 +11,8 @@
 - Calls made before `start()`, a second `start()`, beginning a session that already runs and ending one that does not are now logged at warning level. Failed and non-JSON server answers, and session calls ignored because the SDK Behavior Settings turned session tracking off, are now logged at debug level.
 
 - Mitigated an issue where reaching the event queue threshold logged a warning that events were dropped, although they were sent.
+- Mitigated an issue where a view that was open during a device ID change without merge was ended under the new device ID.
+- Mitigated an issue where session starts let the server work out the user's location from the IP address while the SDK Behavior Settings turned location tracking off.
 
 ## 26.8.1
 - ! Minor breaking change ! Builds now stop with a clear error when the SDK headers find a nlohmann/json older than 3.11 first on the include path, instead of mixing two versions of the library.

CMakeLists.txt

@@ -32,6 +32,7 @@ message("Build Sample programs:" ${COUNTLY_BUILD_SAMPLE})
 set(COUNTLY_PUBLIC_HEADERS
   ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/configuration_module.hpp
   ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/configuration_provider.hpp
+  ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/consent_module.hpp
   ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/constants.hpp
   ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/countly_configuration.hpp
   ${CMAKE_CURRENT_SOURCE_DIR}/include/countly/crash_module.hpp
@@ -57,6 +58,7 @@ add_library(countly
   ${CMAKE_CURRENT_SOURCE_DIR}/src/storage_module_memory.cpp
   ${CMAKE_CURRENT_SOURCE_DIR}/src/event.cpp
   ${CMAKE_CURRENT_SOURCE_DIR}/src/configuration_module.cpp
+  ${CMAKE_CURRENT_SOURCE_DIR}/src/consent_module.cpp
   ${CMAKE_CURRENT_SOURCE_DIR}/src/health_check_module.cpp)
 add_library(countly::countly ALIAS countly)
 
@@ -156,7 +158,8 @@ if(COUNTLY_BUILD_TESTS)
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/internal_limits_integration.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/multi_instance.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/user_details.cpp
-    ${CMAKE_CURRENT_SOURCE_DIR}/tests/health_check.cpp)
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/health_check.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/consent.cpp)
 
     target_compile_options(countly-tests PRIVATE $<$<NOT:$<CXX_COMPILER_ID:MSVC>>:-g>)
     target_compile_definitions(countly-tests PRIVATE COUNTLY_BUILD_TESTS)

include/countly.hpp

@@ -23,6 +23,7 @@
 #ifdef _WIN32
 #undef ERROR
 #endif
+#include "countly/consent_module.hpp"
 #include "countly/event.hpp"
 #include "countly/health_check_module.hpp"
 #include "countly/logger_module.hpp"
@@ -398,6 +399,78 @@ class Countly : public cly::CountlyDelegates {
     configuration->sdkBehaviorSettings = settings_json;
   }
 
+  /**
+   * Requires the user's consent before the SDK collects anything. Each feature then works only after its consent
+   * is given, at start with setConsentEnabled() or later with giveConsent(). The SDK does not remember consent
+   * across restarts: give it again at every start. While the server's SDK behavior settings contain "Consent
+   * required", the server's value applies instead. Call it before start().
+   */
+  void setRequiresConsent(bool required);
+
+  /**
+   * Gives consent at start to the listed features (see cly::consent), and to no other feature. Has an effect only
+   * while consent is required. Names this SDK does not offer are ignored with a warning. Call it before start().
+   */
+  void setConsentEnabled(const std::vector<std::string> &featureNames);
+
+  /**
+   * Gives consent to the listed features (see cly::consent) or feature groups. The features start at once, for
+   * example a session begins with automatic session tracking, and the SDK queues the new consent state for the
+   * server. Ignored while consent is not required. Call it after start().
+   */
+  void giveConsent(const std::vector<std::string> &featureNames);
+
+  /**
+   * Removes consent from the listed features (see cly::consent) or feature groups. The features stop at once:
+   * running views and the session end, and breadcrumbs, user details waiting for the first session and the
+   * location are cleared, the location also on the server. Data recorded before stays queued and is sent. The SDK
+   * queues the new consent state for the server. Ignored while consent is not required. Call it after start().
+   */
+  void removeConsent(const std::vector<std::string> &featureNames);
+
+  /**
+   * Gives consent to every feature this SDK offers, like giveConsent().
+   */
+  void giveConsentAll();
+
+  /**
+   * Removes consent from every feature this SDK offers, like removeConsent().
+   */
+  void removeConsentAll();
+
+  /**
+   * Whether the feature, or every feature of the group, has consent. Always true while consent is not required.
+   */
+  bool getConsent(const std::string &featureName);
+
+  /**
+   * Gives or removes consent for the listed features or feature groups, like giveConsent() and removeConsent().
+   */
+  void setConsent(const std::vector<std::string> &featureNames, bool isConsentGiven);
+
+  /**
+   * Stores a named group of features, so that their consent can be given or removed at once with
+   * setConsentFeatureGroup(), giveConsent() or removeConsent(). Creating a group again replaces it. A group can
+   * not take the name of a feature.
+   */
+  void createFeatureGroup(const std::string &groupName, const std::vector<std::string> &featureNames);
+
+  /**
+   * Gives or removes consent for every feature of the group.
+   */
+  void setConsentFeatureGroup(const std::string &groupName, bool isConsentGiven);
+
+  /**
+   * Logs whether consent is required and whether each feature has consent. Changes nothing.
+   */
+  void checkAllConsent();
+
+  /**
+   * Applies a change of the server's "Consent required" setting: turning the requirement on acts like removing
+   * consent from every feature that has none, lifting it like giving consent to every feature.
+   */
+  void ConsentRequirementChanged() override;
+
 #ifdef COUNTLY_BUILD_TESTS
   /**
    * Convert event queue into list.
@@ -522,6 +595,8 @@ class Countly : public cly::CountlyDelegates {
   /**
    * Updates the session, counted in the health check as a call of the app or, with 'automatic', of the SDK.
    * A call of the app in manual session control without a running session is reported as log code w760.
+   * Without the "sessions" consent a call of the app is ignored, while the SDK's own call still moves the queued
+   * events into a request.
    */
   bool updateSessionInternal(bool automatic);
 
@@ -544,6 +619,73 @@ class Countly : public cly::CountlyDelegates {
    * Stores the health check counters when a session update interval passed since they were last stored.
    */
   void saveHealthCheckOnTimer();
+
+  /**
+   * Where a consent change comes from: a call of the app, the silent removal of a device ID change without merge,
+   * or the server's "Consent required" setting.
+   */
+  enum class ConsentChangeSource { App, DeviceIdChangedWithoutMerge, ServerRequirement };
+
+  /**
+   * Whether the feature has consent; always true while consent is not required.
+   */
+  bool hasConsent(const char *feature) const;
+
+  /**
+   * Whether an event with this key may be recorded: a custom event needs the "events" consent, an internal event
+   * the consent of its own feature, and a "[CLY]_" key that is no known internal event the "events" consent.
+   */
+  bool hasEventConsent(const std::string &key) const;
+
+  /**
+   * Whether location is on: the "location" consent is given and the SDK behavior settings allow location tracking.
+   * While it is off, a session start carries the empty location, which keeps the server from working out the
+   * location from the IP address.
+   */
+  bool isLocationOn();
+
+  /**
+   * The shared part of the public calls that give or remove consent: checks the call, expands groups, records w402
+   * for unknown names and runs the change for the features whose consent changes.
+   */
+  void changeConsent(const std::vector<std::string> &names, bool given, const char *caller);
+
+  /**
+   * Changes the consent of the features: the reactions that need the old consent run first, then the flags change,
+   * the consent state is queued for the server, and the reactions to the new state run. The silent removal of a
+   * device ID change without merge queues no consent state and no location erase. The caller holds
+   * consent_change_mutex and not the instance mutex.
+   */
+  void runConsentChange(const std::vector<std::string> &features, bool given, ConsentChangeSource source);
+
+  /**
+   * The reactions that run while the features still have consent, before it is removed: the running views and the
+   * session end.
+   */
+  void consentWillBeRemoved(const std::vector<std::string> &features);
+
+  /**
+   * The reactions to the new consent of the features. Removed: the breadcrumbs, the user details waiting for the
+   * first session and the stored location are cleared, and a location erase is queued unless the removal is silent.
+   * Given: with automatic sessions a session begins, and the automatic remote config download runs.
+   */
+  void consentChanged(const std::vector<std::string> &features, bool given, ConsentChangeSource source);
+
+  /**
+   * Queues the consent state of every feature this SDK offers.
+   */
+  void queueConsentState();
+
+  /**
+   * Clears the stored location values.
+   */
+  void clearStoredLocation();
+
+  /**
+   * Applies the server's stored "Consent required" setting, and while consent is required, queues the consent
+   * state and, when location is off and no session start will be sent, a location erase. Called at the end of start().
+   */
+  void activateConsent();
   bool began_session = false;
   std::atomic<bool> is_being_disposed{false};
   std::atomic<bool> is_sdk_initialized{false};
@@ -567,6 +709,14 @@ class Countly : public cly::CountlyDelegates {
 
   // Health check counters; created with the instance, so warnings logged before start() are counted too.
   std::shared_ptr<cly::HealthCheckModule> health_check = std::make_shared<cly::HealthCheckModule>();
+
+  // Consent state; created with the instance, so the consent options can be set before start().
+  std::shared_ptr<cly::ConsentModule> consent_module = std::make_shared<cly::ConsentModule>();
+  // Serializes consent changes, their reactions and device ID changes without merge. Taken before the instance mutex.
+  // The destructor takes it once, so that no server requirement change runs while it tears the modules down.
+  std::mutex consent_change_mutex;
+  // Set at the end of start(); before that, a change of the server's requirement only updates the state.
+  bool consent_active = false;
   std::atomic<bool> health_check_sent{false};
   std::chrono::steady_clock::time_point last_health_check_save = std::chrono::steady_clock::now();
 

include/countly/configuration_module.hpp

@@ -40,6 +40,11 @@ class ConfigurationModule : public ConfigurationProvider {
   bool isCustomEventTrackingEnabled();
   bool isCrashReportingEnabled() const override;
 
+  /**
+   * The server's "Consent required" setting: -1 while the SDK behavior settings don't contain it, otherwise 0 or 1.
+   */
+  int getConsentRequired() const;
+
   unsigned int getRequestQueueSizeLimit() const override;
   SDKLimits getLimits() const override;
   unsigned int getEventQueueSizeLimit();

include/countly/consent_module.hpp

@@ -0,0 +1,129 @@
+#ifndef CONSENT_MODULE_HPP_
+#define CONSENT_MODULE_HPP_
+
+#include <map>
+#include <mutex>
+#include <string>
+#include <vector>
+
+#include "nlohmann/json.hpp"
+
+namespace cly {
+/**
+ * The names of the features an app gives or removes consent for, as the server receives them.
+ */
+namespace consent {
+constexpr const char *SESSIONS = "sessions";
+constexpr const char *EVENTS = "events";
+constexpr const char *VIEWS = "views";
+constexpr const char *LOCATION = "location";
+constexpr const char *CRASHES = "crashes";
+constexpr const char *USERS = "users";
+constexpr const char *REMOTE_CONFIG = "remote-config";
+constexpr const char *METRICS = "metrics";
+constexpr const char *FEEDBACK = "feedback";
+constexpr const char *STAR_RATING = "star-rating";
+
+/**
+ * The names of every feature this SDK offers for consent, for example to build a consent screen
+ * or to give every consent at start with setConsentEnabled(cly::consent::allFeatures()).
+ */
+std::vector<std::string> allFeatures();
+} // namespace consent
+
+/**
+ * The consent state of one SDK instance: whether consent is required, which features have consent
+ * and the feature groups the app created. Kept in memory only. Thread safe; it never calls out of
+ * itself while holding its lock.
+ */
+class ConsentModule {
+public:
+  /**
+   * Whether the name is a feature this SDK offers for consent.
+   */
+  static bool isFeature(const std::string &name);
+
+  /**
+   * Sets whether the app requires consent.
+   */
+  void setRequiredByApp(bool required);
+
+  /**
+   * Sets the server's "cr" setting: -1 while the SDK behavior settings don't contain it, otherwise 0 or 1.
+   */
+  void setRequiredByServer(int required);
+
+  /**
+   * Whether the requirement would change if the server's "cr" setting became the given value.
+   */
+  bool requirementChangesWith(int requiredByServer) const;
+
+  /**
+   * Whether consent is required: the server's "cr" while the settings contain it, otherwise the app's choice.
+   */
+  bool isRequired() const;
+
+  /**
+   * Whether the feature, or every feature of the group, has consent. Always true while consent is
+   * not required; false for a name that is neither a feature nor a group.
+   */
+  bool getConsent(const std::string &name) const;
+
+  /**
+   * Whether the name is a feature this SDK offers or a group the app created.
+   */
+  bool isKnownName(const std::string &name) const;
+
+  /**
+   * Whether the name is a group the app created.
+   */
+  bool isGroup(const std::string &name) const;
+
+  /**
+   * Expands feature and group names into feature names, without duplicates and in the order given.
+   * Names that are neither are added to 'unknown'.
+   */
+  std::vector<std::string> expand(const std::vector<std::string> &names, std::vector<std::string> &unknown) const;
+
+  /**
+   * The given features whose own consent flag differs from 'given', regardless of the requirement.
+   */
+  std::vector<std::string> featuresToChange(const std::vector<std::string> &features, bool given) const;
+
+  /**
+   * Sets the consent flag of the given features.
+   */
+  void setGiven(const std::vector<std::string> &features, bool given);
+
+  /**
+   * Gives consent to exactly the given features and removes it from every other one.
+   */
+  void setOnlyGiven(const std::vector<std::string> &features);
+
+  /**
+   * The features whose own consent flag is set, or not set when 'given' is false.
+   */
+  std::vector<std::string> featuresWithFlag(bool given) const;
+
+  /**
+   * The consent flag of every feature this SDK offers, as the consent request sends it.
+   */
+  nlohmann::json state() const;
+
+  /**
+   * Stores a group of features under the name, replacing a group of that name.
+   */
+  void createGroup(const std::string &name, const std::vector<std::string> &features);
+
+private:
+  mutable std::mutex _mutex;
+  bool _requiredByApp = false;
+  int _requiredByServer = -1;
+  std::map<std::string, bool> _given;
+  std::map<std::string, std::vector<std::string>> _groups;
+
+  bool isRequiredLocked() const;
+  bool getConsentLocked(const std::string &name) const;
+};
+} // namespace cly
+#endif

include/countly/constants.hpp

@@ -163,6 +163,11 @@ class CountlyDelegates {
   virtual void RecordEvent(const std::string &key, const std::map<std::string, std::string> &segmentation, int count, double sum, double duration) = 0;
 
   virtual void RecordLocation(const std::string &countryCode, const std::string &city, const std::string &gpsCoordinates, const std::string &ipAddress) = 0;
+
+  /**
+   * Called by the SDK behavior settings when the server's "cr" setting changes whether consent is required.
+   */
+  virtual void ConsentRequirementChanged() {}
 };
 
 } // namespace cly