fix(migrations): drop SQLite columns natively instead of by schema text

[bircni]

The SQLite path of DropTableColumns rebuilt the table from the CREATE TABLE text in sqlite_master and stripped the target columns with a regex that only matched backtick-quoted identifiers. On a database whose schema text was written by another tool, the strip matched nothing, the rebuild copied the column straight across, and the migration still returned success.

Migration 331 drops action_run.concurrency_group that way. Where the drop silently did nothing, the leftover NOT NULL column is one no model writes any more, so every run insert fails and no workflow can start:

[E] repo owner/repo.git: PrepareRunAndInsert: InsertRun: constraint failed: NOT NULL constraint failed: action_run.concurrency_group (1299)

ALTER TABLE ... DROP COLUMN is quoting agnostic and available since SQLite 3.35, well below what either bundled driver ships. Using it also means the index cleanup has to cover composite indexes, not just single-column ones — SQLite refuses to drop a column any index still references, and IDX_action_run_repo_concurrency is exactly such an index.

Migration 349 re-runs the action_run drop, since instances already past 331 do not get another chance at it.

Not backportable to 1.27: 1.27.0 ends at migration ID 342 and 343 is taken by 1.28, so a new migration cannot be added to the release branch. Affected 1.27 instances can drop the two columns by hand in the meantime.

Fixes #38916

[wxiaoguang]

If we have to write these SQLs from scratch .... it's still questionable whether XORM is a production-level framework or it is just a toy: its table DDL functions just can't complete a migration task, developers have to keep writing different logic for different databases ..........

That's also why I don't want to write any new migration for #38921: just bugs, plenty of bugs.

[bircni]

If we have to write these SQLs from scratch .... it's still questionable whether XORM is a production-level framework or it is just a toy: its table DDL functions just can't complete a migration task, developers have to keep writing different logic for different databases ..........

That's also why I don't want to write any new migration for #38921: just bugs, plenty of bugs.

so what shall I do now? xD
I did not introduce xorm and it maybe needs improvements?
But we need to fix this - thats the only way I found

[wxiaoguang]

If we have to write these SQLs from scratch .... it's still questionable whether XORM is a production-level framework or it is just a toy: its table DDL functions just can't complete a migration task, developers have to keep writing different logic for different databases ..........
That's also why I don't want to write any new migration for #38921: just bugs, plenty of bugs.

so what shall I do now? xD I did not introduce xorm and it maybe needs improvements? But we need to fix this - thats the only way I found

I have no idea. Just mention XORM author: @lunny

[bircni]

If we have to write these SQLs from scratch .... it's still questionable whether XORM is a production-level framework or it is just a toy: its table DDL functions just can't complete a migration task, developers have to keep writing different logic for different databases ..........
That's also why I don't want to write any new migration for #38921: just bugs, plenty of bugs.

so what shall I do now? xD I did not introduce xorm and it maybe needs improvements? But we need to fix this - thats the only way I found

I have no idea. Just mention XORM author: @lunny

that repo has over 300 unanswered issues.. And to find it now finally I had to fall over 2 archived Repos lol
https://gitea.com/xorm/xorm

[bircni]

Is there any alternative to xorm?

[bircni]

If we have to write these SQLs from scratch .... it's still questionable whether XORM is a production-level framework or it is just a toy: its table DDL functions just can't complete a migration task, developers have to keep writing different logic for different databases ..........
That's also why I don't want to write any new migration for #38921: just bugs, plenty of bugs.

so what shall I do now? xD I did not introduce xorm and it maybe needs improvements? But we need to fix this - thats the only way I found

I have no idea. Just mention XORM author: @lunny

For this one: the whole table-rebuild dance only exists because SQLite had no DROP COLUMN until 3.35 (2021). We ship 3.53 now.

[TheFox0x7]

Is there any alternative to xorm?

Multiple depending on what you like/need:

• you like writing sql but hate wiring it up in go? sqlc (no mssql, writing raw queries for 4 Dbs in our case + no dynamic queries)
• you like xorm and want something similar? https://gorm.io/ This is probably the closest you'll get
• you prefer something that will result in valid queries and generates from database? https://github.com/go-jet/jet (no mssql)
• you like writing more sql? Raw sql (go 1.27 will add extension to drivers that makes that less annoying afaik but still it's 4x raw sql)
• you prefer modelling it relations in code? https://entgo.io (supposedly bloats the binary through as it uses reflection and disabled dead code optimization - I didn't look into it much though. Also no mssql from what I see)
• Atlas (which you mentioned) is purely for schemas so it's not a replacement for xorm - it might be worth looking into if you want better migrations and don't need any "pro" features

There's also the option of fixing xorm's issues.

[silverwind]

Gorm imho, probably with the new generics api: https://gorm.io/docs/the_generics_way.html

[silverwind]

FYI I've now added a line in 88058a4 in AGENTS.md for this xorm.Sync issue which drops indexes because so far all AIs that I have seen writing migrations have gotten it wrong.