Problem Statement
Planned AI features need to generate embeddings from text (table/column names, SQL queries). The project currently has no dependency on any LLM/embedding API and follows a strict DDD architecture (internal/application/ports/).
Proposed Solution
- Define an
EmbeddingProvider port in internal/application/ports/, e.g. Embed(ctx context.Context, texts []string) ([][]float32, error).
- Implement adapters in
internal/infrastructure/:
- OpenAI (
text-embedding-3-small) for cloud deployments.
- Ollama (
nomic-embed-text or similar) for on-prem/enterprise customers who can't call external APIs.
- Add provider configuration to the YAML config (
config/config.yml and the examples/ samples), following the same pattern as existing database connection config.
- Cache/persist embeddings so they aren't recomputed on every transformation run.
Alternative Solutions
Hardcode a single provider (e.g. OpenAI only) — rejected, since enterprise customers (the project's dual-license model targets them) often require on-prem solutions without external API calls.
Use Case
Shared infrastructure for all downstream AI features (semantic search, rule suggestions, query clustering, text-to-Cypher).
Priority Level
Important
Implementation Ideas
- Follow the existing ports & adapters pattern used for database connections in
internal/infrastructure/persistence/.
- API keys via env vars / Viper config, never hardcoded in YAML.
Additional Context
Related to: "Native vector index foundation" and every feature issue below.
Problem Statement
Planned AI features need to generate embeddings from text (table/column names, SQL queries). The project currently has no dependency on any LLM/embedding API and follows a strict DDD architecture (
internal/application/ports/).Proposed Solution
EmbeddingProviderport ininternal/application/ports/, e.g.Embed(ctx context.Context, texts []string) ([][]float32, error).internal/infrastructure/:text-embedding-3-small) for cloud deployments.nomic-embed-textor similar) for on-prem/enterprise customers who can't call external APIs.config/config.ymland theexamples/samples), following the same pattern as existing database connection config.Alternative Solutions
Hardcode a single provider (e.g. OpenAI only) — rejected, since enterprise customers (the project's dual-license model targets them) often require on-prem solutions without external API calls.
Use Case
Shared infrastructure for all downstream AI features (semantic search, rule suggestions, query clustering, text-to-Cypher).
Priority Level
Important
Implementation Ideas
internal/infrastructure/persistence/.Additional Context
Related to: "Native vector index foundation" and every feature issue below.