SDK development guide fixes

CHANGELOG.md

@@ -1,5 +1,13 @@
 ## XX.XX.XX
 - ! Minor breaking change ! The "Consent required" option of the SDK Behavior Settings now applies. While it is on, the SDK collects nothing until the app gives consent; while it is off, the SDK works as if every consent were given, also in apps that require consent.
+- ! Minor breaking change ! Every request is now sent as POST, with its data in the request body. A client set with `setHTTPClient` gets `usePost` true for every request and must support POST. `alwaysUsePost` is deprecated and has no effect.
+- ! Minor breaking change ! `start()` now refuses a server URL without `http://` or `https://` or without a host, an empty app key and the placeholder `YOUR_APP_KEY`: it logs an error and the SDK stays off. Before, a URL without a scheme was used over plain HTTP.
+- ! Minor breaking change ! Recorded events are now sent on the session update interval (60 seconds by default) or when the event queue threshold is reached, instead of every 3 seconds. `flushEvents()` moves them into a request right away.
+- ! Minor breaking change ! `beginSession`, `updateSession` and `endSession` now work only with `enableManualSessionControl()`; with automatic sessions they are ignored with a warning. A manual `updateSession()` now always reports the session duration, also when called sooner than the session update interval.
+- ! Minor breaking change ! Remote config downloads now use the current remote config request and no longer enroll the user in A/B tests; call `enrollABOnRCDownload()` before `start()` to keep enrolling. With `enableRemoteConfig()`, values are downloaded at the end of `start()`, after a device ID change and when the consent is given, instead of at every session start.
+- ! Minor breaking change ! Config values below their minimum are now ignored with a warning instead of being raised: 0 for the internal limits, `setMaxRequestQueueSize` and `setAutomaticSessionUpdateInterval`, and values below 1 for `setEventsToRQThreshold`. `setEventsToRQThreshold` also keeps values above 10000; the event queue still sends at 1000 events.
+- ! Minor breaking change ! `setLocation` no longer uses values that don't fit their form, such as a lower case country code, coordinates that are not decimal degrees or an invalid IP address; it logs a warning for each.
+- ! Minor breaking change ! SQLite builds now keep the device ID in use between starts, and at `start()` a stored device ID wins: a different device ID given with `setDeviceID` before `start()` is ignored with a warning. To use the given device ID, call `enableClearStoredDeviceID()` before `start()`, or change the device ID with `setDeviceID` after `start()`.
 
 - Added user consent. With `setRequiresConsent(true)` a feature works only after its consent is given, at start with `setConsentEnabled` or later with `giveConsent`, `giveConsentAll` or `setConsent`, and stops when the consent is removed with `removeConsent`, `removeConsentAll` or `setConsent`. The SDK sends every change to the server. The feature names are in `cly::consent`, and `cly::consent::allFeatures()` lists them all. `createFeatureGroup` and `setConsentFeatureGroup` handle groups of features; `getConsent` and `checkAllConsent` show the current state.
 - While consent is required, changing the device ID without merge removes all consent, so the app gives it again for the new user.
@@ -9,10 +17,48 @@
 - Added `disableHealthCheck()` for turning off the health check request.
 - Added `status_code` and `body` to `HTTPResponse`. A custom HTTP client can fill them so that the health check reports the last request the server refused; a client that leaves them empty works as before.
 - Calls made before `start()`, a second `start()`, beginning a session that already runs and ending one that does not are now logged at warning level. Failed and non-JSON server answers, and session calls ignored because the SDK Behavior Settings turned session tracking off, are now logged at debug level.
+- Added device ID generation: when no device ID is stored and the app gives none before `start()`, the SDK generates one. `getDeviceID()` and `getDeviceIDType()` return the device ID in use and where it came from. Every request carries the device ID type (`t`).
+- Added `enableClearStoredDeviceID()` for replacing the stored device ID at `start()` with the given or a newly generated one. The events the stored user left are first sent under the stored device ID. It is meant for testing and forced user resets: left on, every start creates a new user.
+- Added the app version (`av`) and the number of waiting requests (`rr`) to every request.
+- Added the operating system name and version (`_os`, `_os_version`) to the device metrics the SDK sends; values given with `setMetrics` still win.
+- Added event IDs: every event carries its own ID and the current view (`cvid`), custom events the previous custom event (`peid`), view events the previous view (`pvid`) and the platform (`segment`), and crash reports the current view (`_cvid`).
+- Added `setFailedRequestCooldown`: a failed request is tried again after this cool-down, 60 seconds by default, instead of on every update.
+- Added `setRequestDropAgeHours` and the server's "dort" setting for dropping stored requests older than the given age.
+- Added the backoff mechanism: when the server answers slowly, sending pauses for a while. `disableBackoffMechanism()` turns it off, and the SDK Behavior Settings can configure it.
+- Added gradual trimming of a request queue far over its size limit, at most 100 requests at a time; `disableGradualRequestCleaner()` drops the whole overflow at once.
+- Added `setCrashFilterCallback` for changing or dropping crash reports before they are queued, and `setCustomCrashSegmentation` for segmentation added to every crash report.
+- Added `crash().clearBreadcrumbs()`, `views().stopAllViews()`, and segmentation for `closeViewWithName` and `closeViewWithID`.
+- Added `disableLocation()`. A location given with `setLocation` before `start()` is now used from `start()` on.
 
 - Mitigated an issue where reaching the event queue threshold logged a warning that events were dropped, although they were sent.
 - Mitigated an issue where a view that was open during a device ID change without merge was ended under the new device ID.
 - Mitigated an issue where session starts let the server work out the user's location from the IP address while the SDK Behavior Settings turned location tracking off.
+- Mitigated an issue where `start()` threw an exception when the app had set no device ID.
+- Mitigated an issue where calling `views()` or `crash()` before `start()` crashed the app.
+- Mitigated an issue where a log callback that threw an exception broke SDK calls or ended the app.
+- Mitigated an issue where a segmentation whitelist or blacklist in the SDK Behavior Settings removed the names of views, so the views were missing from the dashboard.
+- Mitigated an issue where, while the SDK Behavior Settings turned session tracking off, setting user details or changing the device ID blocked for 15 seconds and then deleted the waiting events, and a running session never ended.
+- Mitigated an issue where any 2xx answer, such as a proxy's HTML page, counted as delivered, so the request was deleted.
+- Mitigated an issue where a negative event count was sent as a very large number. Counts below 1 now become 1, and events with an empty key are ignored, with a warning.
+- Mitigated an issue where an empty device ID was accepted and the server rejected every request.
+- Mitigated an issue where `stop()` left the session open; it now sends the waiting events and ends the session before the SDK stops.
+- Mitigated an issue where sessions were not updated and events were not sent until 100 were waiting when `start()` was called without its last parameter.
+- Mitigated an issue where crash reports lacked the operating system and app version, so the server rejected them. A crash without a stack trace is now ignored with a warning.
+- Mitigated an issue where breadcrumbs stayed after a crash and appeared in the next crash report.
+- Mitigated an issue where an event timed with `startTimer` and `stopTimer` was recorded at its start instead of its end.
+- Mitigated an issue where only the first view after `start()` was marked as the first view of a session, also without a running session.
+- Mitigated an issue where custom view segmentation could replace the view's own keys, and `closeViewWithName` closed any one of several views with that name instead of the last one started.
+- Mitigated an issue where a setting removed from the SDK Behavior Settings kept its last server value until the next start.
+- Mitigated an issue where an empty blacklist in the SDK Behavior Settings switched off the matching whitelist.
+- Mitigated an issue where events with a `[CLY]_` key the SDK does not know skipped the custom event switch, the event filters and the size limits.
+- Mitigated an issue where `setSDKBehaviorSettings` ignored settings given as a copied server answer, and kept the given settings like a server answer for later starts.
+- Mitigated an issue where `disableSDKBehaviorSettingsUpdates()` still let one settings request go out at start.
+- Mitigated an issue where the periodic settings fetch used the device ID from `start()`. The settings are now also fetched after a device ID change.
+- Mitigated an issue where remote config values downloaded for the previous user stayed after a device ID change.
+- Mitigated an issue where a separate location request carried the IP address.
+- Mitigated an issue where values cut to the internal limits were not logged.
+- Mitigated an issue where `setUserDetails` and `setCustomUserDetails` called before `start()` logged that the call was ignored, although the details are sent with the first session.
+- Mitigated an issue where a C++ standard set with `CMAKE_CXX_STANDARD`, for example by a Conan profile, was ignored and the SDK was always compiled as C++14. C++14 stays the default and the minimum.
 
 ## 26.8.1
 - ! Minor breaking change ! Builds now stop with a clear error when the SDK headers find a nlohmann/json older than 3.11 first on the include path, instead of mixing two versions of the library.

CMakeLists.txt

@@ -7,6 +7,11 @@ string(REGEX REPLACE "#define COUNTLY_SDK_VERSION \"([^\"]+)\"" "\\1" COUNTLY_SD
 
 project(countly VERSION ${COUNTLY_SDK_VERSION} LANGUAGES CXX)
 
+# C++14 by default; a newer standard set with CMAKE_CXX_STANDARD is used, an older one is raised to C++14
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+  set(CMAKE_CXX_STANDARD 14)
+endif()
+
 # setup configuration options and their default values
 option(BUILD_SHARED_LIBS "Build shared libraries" ON)
 option(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS  "Create a module definition (.def) on Windows." ON)
@@ -59,7 +64,8 @@ add_library(countly
   ${CMAKE_CURRENT_SOURCE_DIR}/src/event.cpp
   ${CMAKE_CURRENT_SOURCE_DIR}/src/configuration_module.cpp
   ${CMAKE_CURRENT_SOURCE_DIR}/src/consent_module.cpp
-  ${CMAKE_CURRENT_SOURCE_DIR}/src/health_check_module.cpp)
+  ${CMAKE_CURRENT_SOURCE_DIR}/src/health_check_module.cpp
+  ${CMAKE_CURRENT_SOURCE_DIR}/src/platform_info.cpp)
 add_library(countly::countly ALIAS countly)
 
 target_include_directories(countly
@@ -68,9 +74,9 @@ target_include_directories(countly
 
 set_target_properties(countly PROPERTIES
     PUBLIC_HEADER "${COUNTLY_PUBLIC_HEADERS}"
-    CXX_STANDARD 14
     CXX_STANDARD_REQUIRED YES
     CXX_EXTENSIONS NO)
+target_compile_features(countly PRIVATE cxx_std_14)
 
 find_package(Threads)
 target_link_libraries(countly Threads::Threads)
@@ -159,7 +165,15 @@ if(COUNTLY_BUILD_TESTS)
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/multi_instance.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/user_details.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/tests/health_check.cpp
-    ${CMAKE_CURRENT_SOURCE_DIR}/tests/consent.cpp)
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/consent.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/init.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/request_pipeline.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/event_recording.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/session_lifecycle.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/behavior_settings.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/remote_config.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/crash_reporting.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/tests/location.cpp)
 
     target_compile_options(countly-tests PRIVATE $<$<NOT:$<CXX_COMPILER_ID:MSVC>>:-g>)
     target_compile_definitions(countly-tests PRIVATE COUNTLY_BUILD_TESTS)
@@ -181,9 +195,9 @@ if(COUNTLY_BUILD_TESTS)
   target_include_directories(countly-tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/vendor/doctest/doctest)
   target_link_libraries(countly-tests countly)
   set_target_properties(countly-tests PROPERTIES
-    CXX_STANDARD 14
     CXX_STANDARD_REQUIRED YES
     CXX_EXTENSIONS NO)
+  target_compile_features(countly-tests PRIVATE cxx_std_14)
 endif()
 
 if(COUNTLY_BUILD_SAMPLE)
@@ -201,9 +215,9 @@ endif()
   target_include_directories(countly-sample PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/vendor/doctest/doctest)
   target_link_libraries(countly-sample countly)
   set_target_properties(countly-sample PROPERTIES
-    CXX_STANDARD 14
     CXX_STANDARD_REQUIRED YES
     CXX_EXTENSIONS NO)
+  target_compile_features(countly-sample PRIVATE cxx_std_14)
 endif()
 
 install(TARGETS countly

examples/example_integration.cpp

@@ -183,13 +183,12 @@ static void configureAndStart(Countly &instance, const string &appKey, const str
   // Custom HTTP client
   // HTTPClientFunction clientPtr = customClient;
   // instance.setHTTPClient(clientPtr);
-  // instance.alwaysUsePost(true);
   instance.setLogger(printLog);
   instance.SetPath(dbPath); // this will be only built into account if the correct configurations are set
   instance.setDeviceID(deviceId);
   // instance.setSalt("test-salt");
   // OS, OS_version, device, resolution, carrier, app_version);
-  instance.SetMetrics("Windows 10", "10.22", "Mac", "800x600", "Carrier", "1.0");
+  instance.SetMetrics("Windows", "10.0", "Mac", "800x600", "Carrier", "1.0");
 
   instance.setAutomaticSessionUpdateInterval(5); // The value is set so low just for internal validation. Has to be set before start.
   instance.setMaxRQProcessingBatchSize(2);       // in most cases not needed to be set. The value is set so low just for internal validation
@@ -257,7 +256,7 @@ static void stressBothInstances(int threadsPerInstance, int iterations) {
           }
 
           if (i % 5 == 0) {
-            target->updateSession();
+            target->flushEvents();
           }
 
           recorded.fetch_add(2);
@@ -308,7 +307,7 @@ int main() {
     cout << "2) Event with count and sum" << endl;
     cout << "3) Event with count, sum, duration" << endl;
     cout << "4) Event with sum, count, duration and segmentation" << endl;
-    cout << "5) Update Session" << endl;
+    cout << "5) Send waiting events now" << endl;
     cout << "6) Download remote config" << endl;
     cout << "7) Send user detail to server" << endl;
     cout << "8) Change device id with server merge" << endl;
@@ -347,7 +346,7 @@ int main() {
       break;
     }
     case 5:
-      ct.updateSession();
+      ct.flushEvents();
       break;
     case 6: {
       ct.updateRemoteConfig();

examples/qt_demo/README.md

@@ -86,7 +86,6 @@ struct DevConfig {
     std::string sbsJson    = "";
     bool manualSession     = false;
     bool disableSBSUpdates = false;
-    bool alwaysPost        = false;
     bool enableRemoteConfig = false;
     bool disableAutoEventsOnUP = false;
 };

examples/qt_demo/main.cpp

@@ -527,7 +527,6 @@ public slots:
     QTextEdit *sbsJsonEdit = nullptr;
     QCheckBox *manualSessionCheck = nullptr;
     QCheckBox *disableSBSCheck = nullptr;
-    QCheckBox *alwaysPostCheck = nullptr;
     QCheckBox *enableRemoteConfigCheck = nullptr;
     QCheckBox *disableAutoEventsOnUPCheck = nullptr;
     QLineEdit *saltEdit = nullptr;
@@ -758,9 +757,6 @@ public slots:
         disableSBSCheck = new QCheckBox("Disable SBS Updates");
         optLayout->addWidget(disableSBSCheck);
 
-        alwaysPostCheck = new QCheckBox("Always Use POST");
-        optLayout->addWidget(alwaysPostCheck);
-
         enableRemoteConfigCheck = new QCheckBox("Enable Remote Config");
         optLayout->addWidget(enableRemoteConfigCheck);
 
@@ -1478,7 +1474,6 @@ public slots:
         sbsJsonEdit->setText(QString::fromStdString(dc.sbsJson));
         manualSessionCheck->setChecked(dc.manualSession);
         disableSBSCheck->setChecked(dc.disableSBSUpdates);
-        alwaysPostCheck->setChecked(dc.alwaysPost);
         enableRemoteConfigCheck->setChecked(dc.enableRemoteConfig);
         disableAutoEventsOnUPCheck->setChecked(dc.disableAutoEventsOnUP);
         logMsg("[App] Dev config loaded.");
@@ -1497,7 +1492,6 @@ public slots:
         std::string sbsJson = sbsJsonEdit->toPlainText().trimmed().toStdString();
         bool manualSession = manualSessionCheck->isChecked();
         bool disableSBS = disableSBSCheck->isChecked();
-        bool usePost = alwaysPostCheck->isChecked();
 
         if (serverUrl.empty() || appKey.empty()) {
             logMsg("[App] Server URL and App Key are required.");
@@ -1550,11 +1544,6 @@ public slots:
                 logMsg("[App]   SBS JSON provided: " + sbsJson);
             }
 
-            if (usePost) {
-                countly.alwaysUsePost(true);
-                logMsg("[App]   Always POST: enabled");
-            }
-
             if (enableRemoteConfigCheck->isChecked()) {
                 countly.enableRemoteConfig();
                 logMsg("[App]   Remote config: enabled");