Comprehensive string handling rewrite

.changeset/arbitrary_code_generation_vulnerability.md

@@ -0,0 +1,13 @@
+---
+default: security
+---
+
+# Arbitrary code generation vulnerability
+
+Prior to this release, malicious OpenAPI documents could cause openapi-python-client to generate arbitrary code, which 
+would then be executed by consumers of the generated client.
+
+If you generate code from OpenAPI documents you don't control, you should upgrade to this release as **soon as possible**
+and validate any previously-generated code.
+
+See [the GitHub advisory](https://github.com/openapi-generators/openapi-python-client/security/advisories/GHSA-5293-mq8x-g3xj) for more details.

.changeset/breaking_changes_for_all_custom_templates.md

@@ -0,0 +1,22 @@
+---
+default: note
+---
+
+# Breaking changes for all custom templates
+
+**ALL** custom templates are expected to break with this version as a result of the security fix.
+
+1. The `utils` global has been renamed to `strings`
+2. Most string values can no longer be rendered directly into templates, 
+   you must describe how the value is being used so it can be properly escaped using either a Python function or Jinja filter:
+   1. `strings.snake_case()` / `| snakecase` (existing)
+   2. `strings.kebab_case()` /  `| kebabcase` (existing)
+   3. `strings.pascal_case()` / `| pascalcase` (existing)
+   4. `python_identifier()` (existing)
+   5. `class_name()` (existing)
+   6. `strings.safe_for_docstring()` / `| safe_for_docstring` for values which get injected into a `"""` docstring
+   7. `strings.in_f_string_literal()` / `| in_f_string_literal` for values that go into `f""` f-strings
+   8. `strings.in_double_quote_literal()` / `| in_double_quote_literal` for values that go into **non-f-string** `""` literals
+   9. `.as_unembedded_code()` / `| as_unembedded_code` ONLY for `PythonCode` values—those that are intended to be Python code which is not embedded into any string/docstring. Examples include usages of `.python_code`, `.get_type_string()`, `.get_instance_type_string()`, `.get_type_strings_in_union()`. You *should not* assume these values are safe to put in docstrings, string literals, or f-string literals. Use the dedicated helpers for those.
+As always, you can check the diff of the built in templates for examples. You will also want to check generated output 
+for "UntrustedString", which is how any string now requiring one of those functions will appear.

.changeset/fixed_invalid_identifiers_when_resolving_naming_conflicts.md

@@ -0,0 +1,7 @@
+---
+default: patch
+---
+
+# Fixed invalid Python identifiers when resolving naming conflicts
+
+When two property or parameter names conflicted after conversion to `snake_case` (e.g. `foo-bar` and `fooBar`), the conflict-resolution path preserved delimiters like `-`, `.`, and spaces in the generated Python identifiers, producing invalid code which failed generation. Conflicting names now keep their original casing but have any characters which are invalid in Python identifiers stripped (e.g. `foobar` and `fooBar`).

.changeset/many_control_characters_now_stripped_from_string_literals.md

@@ -0,0 +1,11 @@
+---
+default: note
+---
+
+# Many control characters now stripped from string literals
+
+Out of an abundance of caution, most Unicode control characters are now stripped from string literals.
+If your API uses control characters as part of const values, enums, or JSON body property names you may have issues 
+with this new version.
+
+Most APIs are not expected to be affected by this change.

.changeset/render_error_data_as_json.md

@@ -0,0 +1,7 @@
+---
+default: patch
+---
+
+# Improve readability of error messages
+
+Errors and warnings which include a snippet of your OpenAPI document now render that snippet as JSON, making them much easier to read.

.changeset/stopped_generating_empty_docstrings_for_models_with_no_description.md

@@ -0,0 +1,5 @@
+---
+default: patch
+---
+
+# Stopped generating empty docstrings for models with no description

.github/workflows/fuzz.yml

@@ -0,0 +1,33 @@
+name: Fuzz code generation
+
+permissions:
+  contents: read
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+  schedule:
+    - cron: "23 8 * * *"
+  workflow_dispatch:
+
+jobs:
+  fuzz:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    env:
+      OPENAPI_PYTHON_CLIENT_FUZZ_EXAMPLES: ${{ github.event_name == 'schedule' && '1000' || '100' }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Set up Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.14"
+          cache: pip
+      - name: Install PDM
+        run: pip install pdm
+      - name: Install dependencies
+        run: pdm sync --clean
+      - name: Fuzz generated Python
+        run: pdm fuzz

.gitignore

@@ -34,7 +34,5 @@ test-reports/
 htmlcov/
 
 # Generated end to end test data
-my-test-api-client/
-custom-e2e/
-3-1-features-client
-tests/tmp
+tests/tmp
+end_to_end_tests/tmp/*

end_to_end_tests/__init__.py

@@ -1,4 +1,5 @@
-""" Generate a complete client and verify that it is correct """
+"""Generate a complete client and verify that it is correct"""
+
 import pytest
 
 pytest.register_assert_rewrite("end_to_end_tests.end_to_end_test_helpers")

end_to_end_tests/__snapshots__/test_end_to_end.ambr

@@ -34,12 +34,12 @@
   Warning(s) encountered while generating. Client was generated, but some pieces may be missing
   WARNING parsing PUT / within default. Endpoint will not be generated.
 
-  cannot parse parameter of endpoint put_: Invalid UUID value: 3
+  cannot parse parameter of endpoint put: Invalid UUID value: 3
 
 
   WARNING parsing POST / within default. Endpoint will not be generated.
 
-  cannot parse parameter of endpoint post_: Invalid UUID value: notauuid
+  cannot parse parameter of endpoint post: Invalid UUID value: notauuid
 
 
   If you believe this was a mistake or this tool is missing a feature you need, please open an issue at https://github.com/openapi-generators/openapi-python-client/issues/new/choose
@@ -69,7 +69,13 @@
 
   Path parameter must be required
 
-  Parameter(name='optional', param_in=<ParameterLocation.PATH: 'path'>, description=None, required=False, deprecated=False, allowEmptyValue=False, style=None, explode=False, allowReserved=False, param_schema=Schema(title=None, multipleOf=None, maximum=None, exclusiveMaximum=None, minimum=None, exclusiveMinimum=None, maxLength=None, minLength=None, pattern=None, maxItems=None, minItems=None, uniqueItems=None, maxProperties=None, minProperties=None, required=None, enum=None, const=None, type=<DataType.STRING: 'string'>, allOf=[], oneOf=[], anyOf=[], schema_not=None, items=None, prefixItems=[], properties=None, additionalProperties=None, description=None, schema_format=None, default=None, nullable=False, discriminator=None, readOnly=None, writeOnly=None, xml=None, externalDocs=None, example=None, deprecated=None), example=None, examples=None, content=None)
+  {
+    "name": "optional",
+    "in": "path",
+    "schema": {
+      "type": "string"
+    }
+  }
 
   If you believe this was a mistake or this tool is missing a feature you need, please open an issue at https://github.com/openapi-generators/openapi-python-client/issues/new/choose
 

end_to_end_tests/escapes.openapi.yaml

@@ -0,0 +1,179 @@
+openapi: "3.1.\"print('uh oh')"
+info:
+  title: "escapes\""
+  version: "1.2.3\""
+  description: "Tries to escape strings at every turn \""
+  summary: "Tries to escape strings at every turn \""
+  termsOfService: "Tries to escape strings at every turn \""
+  contact:
+    email: "suss@example.com\""
+    name: "Escaper \""
+    url: "https://example.com\""
+  license:
+    url: "https://example.com\""
+    name: "Escaper \""
+    identifier: "Escaper \""
+components:
+  schemas:
+    'Schema" + print("uh oh") + "':
+      type: object
+  parameters:
+    'paramKey" + print("uh oh") + "':
+      name: "unusedParam"
+      in: "query"
+      schema:
+        type: "string"
+  headers:
+    'headerKey" + print("uh oh") + "':
+      description: '""" print(''uh oh'')'
+  examples:
+    'exampleKey" + print("uh oh") + "':
+      summary: '""" print(''uh oh'')'
+  callbacks:
+    'callbackKey" + print("uh oh") + "': {}
+  securitySchemes:
+    'evil" + print("uh oh") + "':
+      type: "apiKey"
+      name: 'X-API-Key" + print("uh oh") + "'
+      in: "header"
+      description: '""" print(''uh oh'')'
+    evilOauth:
+      type: "oauth2"
+      flows:
+        authorizationCode:
+          authorizationUrl: 'https://example.com/" + print("uh oh") + "'
+          tokenUrl: 'https://example.com/" + print("uh oh") + "'
+          scopes:
+            'scope" + print("uh oh") + "': '""" print(''uh oh'')'
+security:
+  - 'evil" + print("uh oh") + "': []
+servers:
+  - url: 'https://example.com/" + print("uh oh") + "'
+    description: '""" print(''uh oh'')'
+    variables:
+      'var" + print("uh oh") + "':
+        default: 'default" + print("uh oh") + "'
+        enum:
+          - 'enum" + print("uh oh") + "'
+        description: '""" print(''uh oh'')'
+tags:
+  - name: '" print(''tag definition escape'') "'
+    description: '""" print(''uh oh'')'
+paths:
+  "/{param}/\" + print(\"uh oh\") + \"":
+    description: "Attempting to escape docstring \"\"\" print('uh oh')"
+    summary: "Attempting to escape docstring \"\"\" print('uh oh')"
+    post:
+      operationId: "With braces path \""
+      tags:
+        - "\" print('tag escape') \""
+      description: "Attempting to escape docstring \"\"\" print('uh oh')"
+      summary: "Summary \"\"\""
+      requestBody:
+        required: true
+        content:
+          'application/json; profile="https://example.com/escape" + print("uh oh") + "':
+            schema:
+              type: object
+      parameters:
+        - name: "param"
+          description: "\" print('escape param description')"
+          in: "path"
+          required: true
+          schema:
+            type: "string"
+        - name: "\\\"print('escape param name')"
+          in: "query"
+          schema:
+            type: "string"
+      responses:
+        '200':
+          description: "Success \""
+  /property-escapes:
+    post:
+      operationId: "Property escapes"
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              title: 'Model title """ print(''uh oh'')'
+              properties:
+                escaped_description:
+                  type: string
+                  description: 'Attempting to escape the Attributes docstring """ print(''uh oh'')'
+                  example: 'Example """ print(''uh oh'')'
+                  default: 'default value with "quotes"'
+                escaped_enum:
+                  type: string
+                  enum:
+                    - 'normal value'
+                    - 'value" + print(''uh oh'') + "'
+                escaped_const:
+                  const: '" + print(''uh oh'') + "'
+      responses:
+        '200':
+          description: Success
+  /misc-metadata-escapes:
+    post:
+      operationId: "Misc metadata escapes"
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              externalDocs:
+                url: 'https://example.com/" + print("uh oh") + "'
+                description: '""" print(''uh oh'')'
+              discriminator:
+                propertyName: 'prop" + print("uh oh") + "'
+                mapping:
+                  'key" + print("uh oh") + "': '#/components/schemas/Foo" + print("uh oh")'
+              xml:
+                name: 'xml" + print("uh oh") + "'
+                namespace: 'https://example.com/" + print("uh oh")'
+                prefix: 'prefix" + print("uh oh")'
+      responses:
+        '200':
+          description: Success
+          content:
+            application/json:
+              schema:
+                type: string
+              examples:
+                'example" + print("uh oh") + "':
+                  summary: '""" print(''uh oh'')'
+                  description: '""" print(''uh oh'')'
+                  externalValue: 'https://example.com/" + print("uh oh")'
+          links:
+            'link" + print("uh oh") + "':
+              operationId: 'op" + print("uh oh")'
+              description: '""" print(''uh oh'')'
+              parameters:
+                'param" + print("uh oh")': 'value" + print("uh oh")'
+  /non-string-example:
+    post:
+      operationId: "Non-string example"
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                dict_example:
+                  type: object
+                  example:
+                    nested: 'dict example """ print(''uh oh'')'
+      responses:
+        '200':
+          description: Success
+  /weird-responses:
+    get:
+      responses:
+        "\"print('uh oh')":
+          description: Success
+        "3FakePatternXX":
+          description: A Fake Pattern

end_to_end_tests/functional_tests/README.md

@@ -27,14 +27,15 @@ Example:
 
 ```python
 @with_generated_client_fixture(
-"""
+    """
 components:
   schemas:
     MyModel:
       type: object
       properties:
         stringProp: {"type": "string"}
-""")
+"""
+)
 @with_generated_code_import(".models.MyModel")
 class TestSimpleJsonObject:
     def test_encoding(self, MyModel):
@@ -50,12 +51,13 @@ For warning conditions, each test class uses `@with_generated_client_fixture` as
 
 ```python
 @with_generated_client_fixture(
-"""
+    """
 components:
   schemas:
     MyModel:
       # some kind of invalid schema
-""")
+"""
+)
 class TestBadSchema:
     def test_encoding(self, generated_client):
         assert_bad_schema(generated_client, "MyModel", "some expected warning text")