diff --git a/README.md b/README.md index 72859a13f..bc6467601 100644 --- a/README.md +++ b/README.md @@ -45,8 +45,7 @@ reflect-cpp and sqlgen fill important gaps in C++ development. They reduce boile - [JSON schema](#json-schema) - [Enums](#enums) - [Algebraic data types](#algebraic-data-types) - - [Extra fields](#extra-fields) - - [std::expected](#stdexpected) + - [Extra fields](#extra-fields) - [Reflective programming](#reflective-programming) - [Standard Library Integration](#support-for-containers) - [The team behind reflect-cpp](#the-team-behind-reflect-cpp) @@ -510,52 +509,6 @@ This results in the following JSON string: {"firstName":"Homer","lastName":"Simpson","age":45,"email":"homer@simpson.com","town":"Springfield"} ``` -### std::expected - -reflect-cpp also supports C++-23's `std::expected`: - -```cpp -#include -#include - -// A success value is serialized like the value itself: -const std::expected age = 45; -const std::string json_string = rfl::json::write(age); -// -> 45 - -// An error is serialized as an object with a single "error" field: -const std::expected no_age = std::unexpected("unknown age"); -const std::string json_string2 = rfl::json::write(no_age); -// -> {"error":"unknown age"} - -const auto age2 = - rfl::json::read>(json_string).value(); -const auto no_age2 = - rfl::json::read>(json_string2).value(); -``` - -`std::expected` can be used anywhere other types can be used, for example as a -field of a struct or inside a container: - -```cpp -struct Person { - std::string first_name; - std::vector> ages; -}; - -const auto homer = - Person{.first_name = "Homer", - .ages = {42, std::unexpected("unknown age")}}; - -const std::string json_string3 = rfl::json::write(homer); -// -> {"first_name":"Homer","ages":[42,{"error":"unknown age"}]} -``` - -`std::expected` requires a standard library that provides the C++-23 feature -(feature-test macro `__cpp_lib_expected`). Note that `std::expected` and -`std::expected` with an identical value and error type are not supported. -Refer to the [documentation](https://rfl.getml.com/expected) for details. - ### Reflective programming Beyond serialization and deserialization, reflect-cpp also supports reflective programming in general. diff --git a/docs/concepts/processors.md b/docs/concepts/processors.md index a0b8cff2c..63880d7e5 100644 --- a/docs/concepts/processors.md +++ b/docs/concepts/processors.md @@ -1,4 +1,4 @@ -# Processors +# Processors Processors can be used to apply transformations to struct serialization and deserialization. @@ -16,10 +16,10 @@ const auto homer = .last_name = "Simpson", .age = 45}; -const auto json_string = +const auto json_string = rfl::json::write(homer); -const auto homer2 = +const auto homer2 = rfl::json::read(json_string).value(); ``` @@ -33,27 +33,28 @@ The resulting JSON string looks like this: reflect-cpp currently supports the following processors: -- `rfl::AddStructName` -- `rfl::AddTagsToVariants` -- `rfl::AddNamespacedTagsToVariants` -- `rfl::AllowRawPtrs` -- `rfl::DefaultIfMissing` -- `rfl::NoExtraFields` -- `rfl::NoFieldNames` -- `rfl::NoOptionals` -- `rfl::UnderlyingEnums` -- `rfl::SnakeCaseToCamelCase` -- `rfl::SnakeCaseToPascalCase` - -### `rfl::AddStructName` +- `rfl::AddStructName` +- `rfl::AddTagsToVariants` +- `rfl::AddNamespacedTagsToVariants` +- `rfl::AllowRawPtrs` +- `rfl::DefaultIfMissing` +- `rfl::EnumNamesOnly` +- `rfl::NoExtraFields` +- `rfl::NoFieldNames` +- `rfl::NoOptionals` +- `rfl::UnderlyingEnums` +- `rfl::SnakeCaseToCamelCase` +- `rfl::SnakeCaseToPascalCase` + +### `rfl::AddStructName` It is also possible to add the struct name as an additional field, like this: ```cpp -const auto json_string = +const auto json_string = rfl::json::write>(homer); -const auto homer2 = +const auto homer2 = rfl::json::read>(json_string).value(); ``` @@ -63,7 +64,7 @@ The resulting JSON string looks like this: {"type":"Person","first_name":"Homer","last_name":"Simpson","age":45} ``` -### `rfl::AddTagsToVariants` +### `rfl::AddTagsToVariants` This processor automatically adds tags to variants. Consider the following example: @@ -150,7 +151,7 @@ const auto msgs = std::vector{ Error::Message{.error = "failure", .error_id = 404} }; -// This would cause problems with rfl::AddTagsToVariants because both +// This would cause problems with rfl::AddTagsToVariants because both // structs have the same name "Message" // But this works perfectly: @@ -203,12 +204,12 @@ This generates: ### `rfl::AllowRawPtrs` -By default, reflect-cpp does not allow *reading into* raw pointers, `std::string_view` or `std::span`. -(*Writing from* raw pointers is never a problem.) This is because reading into raw pointers +By default, reflect-cpp does not allow *reading into* raw pointers, `std::string_view` or `std::span`. +(*Writing from* raw pointers is never a problem.) This is because reading into raw pointers means that the library will allocate memory that the user then has to manually delete. This can lead to misunderstandings and memory leaks. -You might want to consider using some alternatives, such as `std::unique_ptr`, `rfl::Box`, -`std::shared_ptr`, `rfl::Ref` or `std::optional`. +You might want to consider using some alternatives, such as `std::unique_ptr`, `rfl::Box`, +`std::shared_ptr`, `rfl::Ref` or `std::optional`. But if you absolutely have to use raw pointers, you can pass `rfl::AllowRawPtrs` to `read`: ```cpp @@ -253,7 +254,7 @@ if(!person.span.empty()) { The `rfl::DefaultIfMissing` processor is only relevant for reading data. For writing data, it will make no difference. -Usually, when fields are missing in the input data, this will lead to an error +Usually, when fields are missing in the input data, this will lead to an error (unless they are optional fields). But if you pass the `rfl::DefaultIfMissing` processor, then missing fields will be replaced by their default value. @@ -289,13 +290,44 @@ have gotten had you read the following JSON string: Because you have not passed a default value to town, the default value of the type is used instead. +### `rfl::EnumNamesOnly` + +By default, when reading an enum from a string, numeric values are accepted +in addition to the declared enumerator names, even when they do not +correspond to a declared enumerator. For instance, given this enum: + +```cpp +enum class Color { red = 1, green = 2, blue = 3 }; +``` + +reading `{"color":"2"}` will produce `Color::green`, and even +`{"color":"4"}` will succeed and produce the cast value `4`, although `4` +is not a declared enumerator. + +If you want to reject numeric values and only accept the declared +enumerator names, pass the `rfl::EnumNamesOnly` processor to `read`: + +```cpp +const auto circle = + rfl::json::read(json_str); +``` + +Now, `{"color":"2"}` will lead to an error: + +``` +Failed to parse field 'color': Invalid enum value: '2'. Must be one of [red, green, blue]. +``` + +This processor only affects reading. Enum values that cannot be matched to +a declared name are still written as their integer representation. + ### `rfl::NoExtraFields` -When reading an object and the object contains a field that cannot be +When reading an object and the object contains a field that cannot be matched to any of the fields in the struct, that field is simply ignored. However, when `rfl::NoExtraFields` is added to `read`, then such extra fields -will lead to an error. +will lead to an error. This can be overriden by adding `rfl::ExtraFields` to the struct. @@ -312,7 +344,7 @@ struct Person { {"first_name":"Homer","last_name":"Simpson","extra_field":0} ``` -If you call `rfl::json::read(json_string)`, then `extra_field` will +If you call `rfl::json::read(json_string)`, then `extra_field` will simply be ignored. But if you call `rfl::json::read(json_string)`, @@ -333,13 +365,13 @@ will not fail, because `extra_field` would be included in `extras`. ### `rfl::NoFieldNames` -We can also remove the field names altogether: +We can also remove the field names altogether: ```cpp -const auto json_string = +const auto json_string = rfl::json::write(homer); -const auto homer2 = +const auto homer2 = rfl::json::read(json_string).value(); ``` @@ -351,19 +383,19 @@ The resulting JSON string looks like this: This is particularly relevant for binary formats, which do not emphasize readability, like msgpack or flexbuffers. Removing the field names can reduce the size of the -resulting bytestrings and significantly speed up read and write time, +resulting bytestrings and significantly speed up read and write time, depending on the dataset. However, it makes it more difficult to maintain backwards compatability. Note that `rfl::NoFieldNames` is not supported for BSON, TOML, XML, or YAML, due -to limitations of these formats. +to limitations of these formats. ### `rfl::NoOptionals` As we have seen in the section on optional fields, when a `std::optional` is `std::nullopt`, it is usually not written at all. But if you want them to be explicitly -written as `null`, you can use this processor. The same thing applies to `std::shared_ptr` and +written as `null`, you can use this processor. The same thing applies to `std::shared_ptr` and `std::unique_ptr`. ```cpp @@ -384,7 +416,7 @@ The resulting JSON string looks like this: {"first_name":"Homer","last_name":"Simpson","town":null} ``` -By default, `rfl::json::read` will accept both `"town":null` and just +By default, `rfl::json::read` will accept both `"town":null` and just leaving out the field `town`. However, if you want to require the field `town` to be included, you can add `rfl::NoOptionals` to `read`: @@ -424,10 +456,10 @@ Please refer to the example above. If you want `PascalCase` instead of `camelCase`, you can use the appropriate processor: ```cpp -const auto json_string = +const auto json_string = rfl::json::write(homer); -const auto homer2 = +const auto homer2 = rfl::json::read(json_string).value(); ``` @@ -442,10 +474,10 @@ The resulting JSON string looks like this: You can combine several processors: ```cpp -const auto json_string = +const auto json_string = rfl::json::write>(homer); -const auto homer2 = +const auto homer2 = rfl::json::read>(json_string).value(); ``` diff --git a/docs/enums.md b/docs/enums.md index 849b24889..5765fc261 100644 --- a/docs/enums.md +++ b/docs/enums.md @@ -164,6 +164,22 @@ This will be represented as follows: This works, because 16 + 256 + 512 + 1024 + 8192 = 10000. Flag enums are *always* represented in terms of 2^N-numbers. +## Reading numeric values as enums + +When reading, enum values can also be given as numbers. For instance, given this enum: + +```cpp +enum class Color { red = 1, green = 2, blue = 3 }; +``` + +reading `{"color":"2"}` will produce `Color::green`. + +By default, this also works when the number does not correspond to a +declared enumerator (for instance, `{"color":"4"}` will produce the cast +value `4`). If you want to reject numeric values and only accept the +declared enumerator names, pass the +[`rfl::EnumNamesOnly`](concepts/processors.md) processor to `read`. + ## General-purpose enumeration utilities reflect-cpp also allows you to directly convert between enumerator values and strings: diff --git a/include/rfl.hpp b/include/rfl.hpp index 47789a4e8..68e355886 100644 --- a/include/rfl.hpp +++ b/include/rfl.hpp @@ -21,6 +21,7 @@ #include "rfl/DefaultIfMissing.hpp" #include "rfl/DefaultVal.hpp" #include "rfl/Description.hpp" +#include "rfl/EnumNamesOnly.hpp" #include "rfl/ExtraFields.hpp" #include "rfl/Field.hpp" #include "rfl/Flatten.hpp" diff --git a/include/rfl/EnumNamesOnly.hpp b/include/rfl/EnumNamesOnly.hpp new file mode 100644 index 000000000..6d66bee6b --- /dev/null +++ b/include/rfl/EnumNamesOnly.hpp @@ -0,0 +1,31 @@ +#ifndef RFL_ENUMNAMESONLY_HPP_ +#define RFL_ENUMNAMESONLY_HPP_ + +namespace rfl { + +/// A processor that instructs parsers to accept only the declared names of an +/// enum's enumerators when reading an enum from a string. +/// This is a marker type (doesn't modify data) that changes parser behavior. +/// By default, when reading an enum from a string, numeric values are accepted +/// in addition to the declared enumerator names, even when they do not +/// correspond to a declared enumerator (for instance, reading "4" into an enum +/// with values 1, 2 and 3 will produce the cast value 4). +/// When EnumNamesOnly is added as a processor, numeric values are rejected and +/// only the declared enumerator names will be accepted. +/// Usage: rfl::json::read(json_str) +struct EnumNamesOnly { + public: + /// Identity process function - returns the named tuple unchanged. + /// The actual validation happens in the parser, not here. + /// @tparam StructType The struct type being processed + /// @param _named_tuple The named tuple representation of the struct + /// @return The same named tuple (unchanged) + template + static auto process(auto&& _named_tuple) { + return _named_tuple; + } +}; + +} // namespace rfl + +#endif diff --git a/include/rfl/enums.hpp b/include/rfl/enums.hpp index 33975f9b7..fd28815c8 100644 --- a/include/rfl/enums.hpp +++ b/include/rfl/enums.hpp @@ -81,28 +81,44 @@ std::string enum_to_string(const EnumType _enum) { } // Converts a string to a value of the given enum type. -template +// +// By default, numeric values are accepted in addition to the declared +// enumerator names, even when they do not correspond to a declared +// enumerator (for instance, reading "4" into an enum with values 1, 2 and 3 +// will produce the cast value 4). Pass enum_names_only = true (or use the +// EnumNamesOnly processor with a parser) to accept only the declared +// enumerator names instead. +template Result string_to_enum(const std::string& _str) { + const auto make_error_msg = [&](const auto& name) { + std::string msg = "Invalid enum value: '"; + msg += name; + msg += "'. Must be one of ["; + const char* sep = ""; + for (const auto& p : get_enumerator_array()) { + msg += sep; + msg += p.first; + sep = ", "; + } + msg += "]."; + return error(msg); + }; + const auto cast_numbers_or_names = - [](const std::string& name) -> Result { + [&](const std::string& name) -> Result { const auto r = internal::enums::from_string(name); if (r) { return *r; } - try { - return static_cast(std::stoi(name)); - } catch (std::exception& exp) { - std::string msg = "Invalid enum value: '"; - msg += name; - msg += "'. Must be one of ["; - const char* sep = ""; - for (const auto& p : get_enumerator_array()) { - msg += sep; - msg += p.first; - sep = ", "; + if constexpr (enum_names_only) { + return make_error_msg(name); + } else { + try { + const auto val = std::stoi(name); + return static_cast(val); + } catch (std::exception& exp) { + return make_error_msg(name); } - msg += "]."; - return error(msg); } }; diff --git a/include/rfl/internal/enum_names_only_v.hpp b/include/rfl/internal/enum_names_only_v.hpp new file mode 100644 index 000000000..4fc562080 --- /dev/null +++ b/include/rfl/internal/enum_names_only_v.hpp @@ -0,0 +1,32 @@ +#ifndef RFL_INTERNAL_ENUMNAMESONLY_HPP_ +#define RFL_INTERNAL_ENUMNAMESONLY_HPP_ + +#include + +#include "../EnumNamesOnly.hpp" +#include "../Processors.hpp" + +namespace rfl::internal { + +template +class enum_names_only; + +template +class enum_names_only : public std::false_type {}; + +template <> +class enum_names_only : public std::true_type {}; + +template +struct enum_names_only> { + static constexpr bool value = + (enum_names_only::value || ... || enum_names_only::value); +}; + +template +constexpr bool enum_names_only_v = + enum_names_only>>::value; + +} // namespace rfl::internal + +#endif diff --git a/include/rfl/parsing/ParserEnum.hpp b/include/rfl/parsing/ParserEnum.hpp index 174d2cdcd..4a216f434 100644 --- a/include/rfl/parsing/ParserEnum.hpp +++ b/include/rfl/parsing/ParserEnum.hpp @@ -7,6 +7,7 @@ #include "../Result.hpp" #include "../config.hpp" #include "../enums.hpp" +#include "../internal/enum_names_only_v.hpp" #include "../internal/enums/is_flag_enum.hpp" #include "../internal/enums/is_scoped_enum.hpp" #include "../internal/has_reflector.hpp" @@ -60,7 +61,8 @@ struct ParserEnum { .transform([](const auto _val) { return static_cast(_val); }); } else { return _r.template to_basic_type(_var).and_then( - rfl::string_to_enum); + rfl::string_to_enum>); } } diff --git a/tests/json/test_enum_names_only.cpp b/tests/json/test_enum_names_only.cpp new file mode 100644 index 000000000..fdd73af34 --- /dev/null +++ b/tests/json/test_enum_names_only.cpp @@ -0,0 +1,152 @@ +#include + +#include +#include +#include + +namespace test_enum_names_only { + +enum class Color { red = 1, green = 2, blue = 3 }; + +enum class Perm { + read = 1, + write = 2, + exec = 4 +}; + +inline Perm operator|(Perm a, Perm b) noexcept { + return static_cast(static_cast(a) | static_cast(b)); +} + +// An enum with a narrow underlying type. +enum class Narrow : uint8_t { low = 1, high = 2 }; + +struct Circle { + float radius; + Color color; +}; + +struct File { + int id; + Perm perm; +}; + +TEST(json, test_numeric_enum_allowed_by_default) { + // Legacy behavior: numeric values are accepted as cast values, even when + // they do not correspond to a declared enumerator. + const auto res = rfl::json::read(R"({"radius":2.0,"color":"4"})"); + ASSERT_TRUE(res && true) << "Test failed on read. Error: " + << res.error().what(); + EXPECT_EQ(static_cast(res.value().color), 4); +} + +TEST(json, test_enum_name_allowed_by_default) { + const auto res = rfl::json::read(R"({"radius":2.0,"color":"red"})"); + ASSERT_TRUE(res && true) << "Test failed on read. Error: " + << res.error().what(); + EXPECT_EQ(res.value().color, Color::red); +} + +TEST(json, test_unknown_enum_name_rejected_by_default) { + const auto res = rfl::json::read(R"({"radius":2.0,"color":"bart"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'color': Invalid enum value: 'bart'. Must be one of [red, green, blue].)"); +} + +TEST(json, test_flag_enum_numeric_allowed_by_default) { + const auto res = rfl::json::read(R"({"id":1,"perm":"read|8"})"); + ASSERT_TRUE(res && true) << "Test failed on read. Error: " + << res.error().what(); + EXPECT_EQ(static_cast(res.value().perm), 9); +} + +TEST(json, test_enum_name_allowed_with_processor) { + const auto res = + rfl::json::read( + R"({"radius":2.0,"color":"red"})"); + ASSERT_TRUE(res && true) << "Test failed on read. Error: " + << res.error().what(); + EXPECT_EQ(res.value().color, Color::red); +} + +TEST(json, test_declared_numeric_value_rejected_with_processor) { + // Numeric values are rejected even when they correspond to a declared + // enumerator: only the declared enumerator names are accepted. + const auto res = + rfl::json::read( + R"({"radius":2.0,"color":"2"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'color': Invalid enum value: '2'. Must be one of [red, green, blue].)"); +} + +TEST(json, test_undeclared_numeric_value_rejected_with_processor) { + const auto res = + rfl::json::read( + R"({"radius":2.0,"color":"4"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'color': Invalid enum value: '4'. Must be one of [red, green, blue].)"); +} + +TEST(json, test_negative_numeric_value_rejected_with_processor) { + const auto res = + rfl::json::read( + R"({"radius":2.0,"color":"-1"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'color': Invalid enum value: '-1'. Must be one of [red, green, blue].)"); +} + +TEST(json, test_unknown_enum_name_rejected_with_processor) { + const auto res = + rfl::json::read( + R"({"radius":2.0,"color":"bart"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'color': Invalid enum value: 'bart'. Must be one of [red, green, blue].)"); +} + +TEST(json, test_flag_enum_names_allowed_with_processor) { + const auto res = + rfl::json::read( + R"({"id":1,"perm":"read|write"})"); + ASSERT_TRUE(res && true) << "Test failed on read. Error: " + << res.error().what(); + EXPECT_EQ(res.value().perm, Perm::read | Perm::write); +} + +TEST(json, test_flag_enum_numeric_token_rejected_with_processor) { + const auto res = + rfl::json::read(R"({"id":1,"perm":"read|8"})"); + EXPECT_TRUE(!res.has_value()); + EXPECT_EQ(res.error().what(), + R"(Failed to parse field 'perm': Invalid enum value: '8'. Must be one of [read, write, exec].)"); +} + +TEST(json, test_string_to_enum_direct) { + // With enum_names_only = true, only the declared names are accepted. + const auto name = rfl::string_to_enum("green"); + ASSERT_TRUE(name.has_value()); + EXPECT_EQ(*name, Color::green); + const auto declared_numeric = rfl::string_to_enum("2"); + EXPECT_TRUE(!declared_numeric.has_value()); + EXPECT_EQ(declared_numeric.error().what(), + R"(Invalid enum value: '2'. Must be one of [red, green, blue].)"); + const auto undeclared_numeric = rfl::string_to_enum("4"); + EXPECT_TRUE(!undeclared_numeric.has_value()); + // Values that overflow the narrow underlying type are rejected as well. + const auto narrow = rfl::string_to_enum("300"); + EXPECT_TRUE(!narrow.has_value()); + // The default (permissive) behavior is unchanged: numeric values are + // accepted as cast values. + const auto permissive = rfl::string_to_enum("4"); + ASSERT_TRUE(permissive.has_value()); + EXPECT_EQ(static_cast(*permissive), 4); + const auto permissive_narrow = rfl::string_to_enum("300"); + ASSERT_TRUE(permissive_narrow.has_value()); + EXPECT_EQ(static_cast(*permissive_narrow), 44); +} + +} // namespace test_enum_names_only