Repository navigation
fix(purchase): refuse a GCP CUD when one for the same project, region and family started in the last 24 hours (Part of #682) #1848
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cristim
wants to merge
4
commits into
main
Choose a base branch
from
fix/682-p1-gcp-recent-check
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
edff79b
fix(purchase): refuse a GCP CUD when one for the same project, region…
cristim a96345d
test(purchase): pin that recent commitments from two projects fail cl…
cristim 2eb600b
Merge remote-tracking branch 'origin/main' into fix/682-p1-gcp-recent…
cristim 903695f
test(purchase): pin that another lineage's recent CUD is refused, not…
cristim File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| package purchase | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
| "fmt" | ||
| "slices" | ||
|
|
||
| "github.com/LeanerCloud/cloud-commitments-go/pkg/common" | ||
| "github.com/LeanerCloud/cloud-commitments-go/pkg/logging" | ||
| "github.com/LeanerCloud/cloud-commitments-go/pkg/provider" | ||
| "github.com/LeanerCloud/cloud-commitments-go/pkg/recfilter" | ||
|
|
||
| "github.com/LeanerCloud/cloud-commitments-platform/internal/config" | ||
| ) | ||
|
|
||
| // Fixed refusal notes (platform#682). Both are stored verbatim and are on the | ||
| // stored-error allowlist (errnotes.go), so neither may carry provider text. | ||
| const ( | ||
| gcpDuplicateCheckFailedNote = "GCP duplicate check could not list existing commitments; no purchase was submitted" | ||
| gcpRecentCommitmentNote = "a GCP commitment for this project, region and machine family started in the last 24 hours; no purchase was submitted, to avoid a duplicate" | ||
| ) | ||
|
|
||
| // gcpCommitmentNameTokenLen mirrors computeengine idempotentNameTokenLen | ||
| // (providers/gcp/services/computeengine/client.go:1411). | ||
| const gcpCommitmentNameTokenLen = 32 | ||
|
|
||
| // gcpCommitmentName mirrors computeengine idempotentCommitmentName | ||
| // (client.go:1423-1434) for a lowercase hex token: "cud-" + token[:32]. The | ||
| // parity test drives the real client, so a rule change there fails it. | ||
| func gcpCommitmentName(token string) string { | ||
| if len(token) > gcpCommitmentNameTokenLen { | ||
| token = token[:gcpCommitmentNameTokenLen] | ||
| } | ||
| return "cud-" + token | ||
| } | ||
|
|
||
| func isGCPCompute(providerName string, service common.ServiceType) bool { | ||
| return common.ProviderType(providerName) == common.ProviderGCP && service == common.ServiceCompute | ||
| } | ||
|
|
||
| // gcpOwnCommitmentNames returns the commitment names this execution's selected | ||
| // GCP compute recs will create, built from the same per-rec tokens the purchase | ||
| // path passes to the provider. A listed commitment with one of these names was | ||
| // created by this lineage (a lost-response re-drive, or an earlier rec of the | ||
| // same run), so it is not a duplicate of the rec being bought. | ||
| func gcpOwnCommitmentNames(exec *config.PurchaseExecution, selected []int) map[string]bool { | ||
| names := make(map[string]bool) | ||
| for _, i := range selected { | ||
| rec := exec.Recommendations[i] | ||
| service, _ := mapServiceSlug(rec.Service) | ||
| if !isGCPCompute(rec.Provider, service) { | ||
| continue | ||
| } | ||
| names[gcpCommitmentName(common.DeriveIdempotencyToken(idempotencyLineageKey(exec), i))] = true | ||
| } | ||
| return names | ||
| } | ||
|
|
||
| // recentCommitmentFilter is the optional GCP filter recfilter asserts on the | ||
| // client it lists through (pkg/recfilter/dedupe.go:16-18, :71). | ||
| type recentCommitmentFilter interface { | ||
| FilterRecommendationsForRecentCommitments(recs []common.Recommendation, existing []common.Commitment) (passed, filtered []common.Recommendation, err error) | ||
| } | ||
|
|
||
| // gcpRecentDedupeClient lists without this execution's own commitments and | ||
| // forwards the GCP filter. Embedding the interface does not promote | ||
| // FilterRecommendationsForRecentCommitments, and without it recfilter would | ||
| // fall back to its generic map path, which does not understand CUDs. | ||
| type gcpRecentDedupeClient struct { | ||
| provider.ServiceClient | ||
| own map[string]bool | ||
| } | ||
|
|
||
| func (c gcpRecentDedupeClient) GetExistingCommitments(ctx context.Context) ([]common.Commitment, error) { | ||
| all, err := c.ServiceClient.GetExistingCommitments(ctx) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return slices.DeleteFunc(slices.Clone(all), func(cm common.Commitment) bool { return c.own[cm.CommitmentID] }), nil | ||
| } | ||
|
|
||
| // FilterRecommendationsForRecentCommitments stamps the project onto the recs | ||
| // from the listed commitments (each carries Account = the client's project) and | ||
| // delegates. The platform does not know the project in ADC mode, so the rec | ||
| // cannot carry it itself. | ||
| func (c gcpRecentDedupeClient) FilterRecommendationsForRecentCommitments(recs []common.Recommendation, existing []common.Commitment) (passed, filtered []common.Recommendation, err error) { | ||
| inner, ok := c.ServiceClient.(recentCommitmentFilter) | ||
| if !ok { | ||
| return nil, nil, errors.New("GCP service client has no recent-commitment filter") | ||
| } | ||
| if len(existing) == 0 { | ||
| return nil, nil, errors.New("no recent commitments to filter against") | ||
| } | ||
| project := existing[0].Account | ||
| for i := range existing { | ||
| if existing[i].Account != project { | ||
| return nil, nil, fmt.Errorf("recent commitments span projects %q and %q", project, existing[i].Account) | ||
| } | ||
| } | ||
| stamped := slices.Clone(recs) | ||
| for i := range stamped { | ||
| stamped[i].Account = project | ||
| } | ||
| return inner.FilterRecommendationsForRecentCommitments(stamped, existing) | ||
| } | ||
|
|
||
| // gcpRecentCommitmentRefusal refuses a GCP compute rec when the project already | ||
| // has a commitment of the rec's region and machine family that started in the | ||
| // last 24 hours, and fails closed when the commitments cannot be listed. It | ||
| // returns nil for every other rec. A refusal is a beforeSubmission error: the | ||
| // purchase call is never made. | ||
| func gcpRecentCommitmentRefusal(ctx context.Context, client provider.ServiceClient, rec common.Recommendation, ownNames map[string]bool) error { | ||
| if !isGCPCompute(string(rec.Provider), rec.Service) { | ||
| return nil | ||
| } | ||
| rec.CommitmentType = common.CommitmentCUD | ||
| passed, filtered, err := recfilter.NewDuplicateChecker(0).AdjustRecommendationsForExisting( | ||
| ctx, []common.Recommendation{rec}, gcpRecentDedupeClient{ServiceClient: client, own: ownNames}) | ||
| switch { | ||
| case err != nil: | ||
| logging.Errorf("GCP duplicate check for %s in %s failed: %v", rec.ResourceType, rec.Region, err) | ||
| return beforeSubmission(errors.New(gcpDuplicateCheckFailedNote)) | ||
| case len(filtered) > 0: | ||
| return beforeSubmission(errors.New(gcpRecentCommitmentNote)) | ||
| case len(passed) != 1: | ||
| logging.Errorf("GCP duplicate check for %s in %s returned %d passed recs, want 1", rec.ResourceType, rec.Region, len(passed)) | ||
| return beforeSubmission(errors.New(gcpDuplicateCheckFailedNote)) | ||
| } | ||
| return nil | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Script executed:
Repository: LeanerCloud/cloud-commitments-platform
Length of output: 11501
Script executed:
Repository: LeanerCloud/cloud-commitments-platform
Length of output: 42433
Script executed:
Repository: LeanerCloud/cloud-commitments-platform
Length of output: 42895
Script executed:
Repository: LeanerCloud/cloud-commitments-platform
Length of output: 43513
Script executed:
Repository: LeanerCloud/cloud-commitments-platform
Length of output: 33140
Reconcile the existing commitment before refusing the retry.
If a retry lists both its own commitment and a recent matching foreign commitment, the foreign commitment blocks adoption. The retry can then report an existing purchase as unpurchased. Reconcile the verified own commitment first, while retaining the refusal when no matching own commitment exists.