generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 23
Add SARIF output format support with comprehensive tests #290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
5c43184
Add SARIF output format support with comprehensive tests
tautschnig 09ba853
Address feedback
tautschnig 6515425
Update StrataTest/Languages/Boogie/SarifOutputTests.lean
tautschnig 5e6f19e
Update Strata/Languages/Boogie/SarifOutput.lean
tautschnig 763c240
Update StrataVerify.lean
tautschnig c5cfe82
Refactor to address Aaron's comments
tautschnig feac468
Merge remote-tracking branch 'origin/main' into tautschnig/SARIF-output
tautschnig 636225f
Fixup merge after #296 got merged in
tautschnig db83267
Merge remote-tracking branch 'origin/main' into tautschnig/SARIF-output
tautschnig 4c10bb1
Merge remote-tracking branch 'origin/main' into tautschnig/SARIF-output
tautschnig 248beb7
Update to latest main
tautschnig bf37a14
Merge branch 'main' into tautschnig/SARIF-output
shigoel e452315
Update StrataTest/Languages/Boogie/SarifOutputTests.lean
tautschnig 1b345bc
Address comments
tautschnig 5caead0
Merge remote-tracking branch 'origin/main' into tautschnig/SARIF-output
tautschnig 4a4fb61
Fixup merge
tautschnig 43ce2bf
Test fixes
tautschnig 4709802
Merge remote-tracking branch 'origin/main' into tautschnig/SARIF-output
tautschnig fbac11d
Merge fix-up
tautschnig 121d58d
Update StrataTest/Languages/Core/SarifOutputTests.lean
tautschnig 2768771
Fix remaining Boogie reference
tautschnig 7464e60
Merge branch 'main' into tautschnig/SARIF-output
shigoel 6c662c1
Adjust Boogie -> Core in example
tautschnig aee6ddd
Merge branch 'main' into tautschnig/SARIF-output
shigoel 2a070a3
Merge branch 'main' into tautschnig/SARIF-output
shigoel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| program Core; | ||
|
|
||
| // Simple test program for SARIF output | ||
| procedure AddPositive(x : int, y : int) returns (z : int) | ||
| spec { | ||
| requires [x_positive]: (x > 0); | ||
| requires [y_positive]: (y > 0); | ||
| ensures [result_positive]: (z > 0); | ||
| ensures [result_sum]: (z == x + y); | ||
| } | ||
| { | ||
| z := x + y; | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tautschnig marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| /- | ||
| Copyright Strata Contributors | ||
|
|
||
| SPDX-License-Identifier: Apache-2.0 OR MIT | ||
| -/ | ||
|
|
||
| import Strata.Languages.Core.Verifier | ||
| import Strata.Util.Sarif | ||
|
|
||
| /-! | ||
| # Core SARIF Output | ||
|
|
||
| This module provides Core-specific conversion functions for SARIF output. | ||
| -/ | ||
|
|
||
| namespace Core.Sarif | ||
|
|
||
| open Strata.Sarif Strata.SMT | ||
|
|
||
| /-! ## Core-Specific Conversion Functions -/ | ||
|
|
||
| /-- Convert Core Outcome to SARIF Level -/ | ||
| def outcomeToLevel : Outcome → Level | ||
| | .pass => .none | ||
| | .fail => .error | ||
| | .unknown => .warning | ||
| | .implementationError _ => .error | ||
|
|
||
| /-- Convert Core Outcome to a descriptive message -/ | ||
| def outcomeToMessage (outcome : Outcome) (smtResult : SMT.Result) : String := | ||
| match outcome with | ||
| | .pass => "Verification succeeded" | ||
| | .fail => | ||
| match smtResult with | ||
| | .sat m => | ||
| if m.isEmpty then | ||
| "Verification failed" | ||
| else | ||
| s!"Verification failed with counterexample: {Std.format m}" | ||
| | _ => "Verification failed" | ||
| | .unknown => "Verification result unknown (solver timeout or incomplete)" | ||
| | .implementationError msg => s!"Verification error: {msg}" | ||
|
|
||
| /-- Extract location information from metadata -/ | ||
| def extractLocation (md : Imperative.MetaData Expression) : Option Location := do | ||
| let fileRangeElem ← md.findElem Imperative.MetaData.fileRange | ||
| match fileRangeElem.value with | ||
| | .file2dRange fr => | ||
| let uri := match fr.file with | ||
| | .file path => path | ||
| pure { uri, startLine := fr.start.line, startColumn := fr.start.column } | ||
| | _ => none | ||
|
|
||
| /-- Convert a VCResult to a SARIF Result -/ | ||
| def vcResultToSarifResult (vcr : VCResult) : Strata.Sarif.Result := | ||
| let ruleId := vcr.obligation.label | ||
| let level := outcomeToLevel vcr.result | ||
| let messageText := outcomeToMessage vcr.result vcr.smtResult | ||
| let message : Strata.Sarif.Message := { text := messageText } | ||
|
|
||
| let locations := match extractLocation vcr.obligation.metadata with | ||
| | some loc => #[locationToSarif loc] | ||
| | none => #[] | ||
|
|
||
| { ruleId, level, message, locations } | ||
|
|
||
| /-- Convert VCResults to a SARIF document -/ | ||
| def vcResultsToSarif (vcResults : VCResults) : Strata.Sarif.SarifDocument := | ||
| let tool : Strata.Sarif.Tool := { | ||
| driver := { | ||
| name := "Strata", | ||
| version := "0.1.0", | ||
| informationUri := "https://github.com/strata-org/Strata" | ||
| } | ||
| } | ||
|
|
||
| let results := vcResults.map vcResultToSarifResult | ||
|
|
||
| let run : Strata.Sarif.Run := { tool, results } | ||
|
|
||
| { version := "2.1.0", | ||
| schema := "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json", | ||
| runs := #[run] } | ||
|
|
||
| end Core.Sarif |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| /- | ||
| Copyright Strata Contributors | ||
|
|
||
| SPDX-License-Identifier: Apache-2.0 OR MIT | ||
| -/ | ||
|
|
||
| import Lean.Data.Json | ||
|
|
||
| /-! | ||
| # SARIF Output | ||
|
|
||
| This module provides support for outputting results in SARIF | ||
| (Static Analysis Results Interchange Format) version 2.1.0. | ||
|
|
||
| SARIF specification: https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html | ||
| -/ | ||
|
|
||
| namespace Strata.Sarif | ||
|
|
||
| open Lean (Json ToJson FromJson) | ||
|
|
||
| /-! ## SARIF Data Structures -/ | ||
|
|
||
| /-- SARIF location representing a position in source code -/ | ||
| structure Location where | ||
| uri : String | ||
| startLine : Nat | ||
| startColumn : Nat | ||
| deriving Repr, ToJson, FromJson, BEq | ||
atomb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| /-- SARIF artifact location representing a file URI -/ | ||
| structure ArtifactLocation where | ||
| uri : String | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| /-- SARIF region representing a source code region with line and column information -/ | ||
| structure Region where | ||
| startLine : Nat | ||
| startColumn : Nat | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| /-- SARIF physical location with region information -/ | ||
| structure PhysicalLocation where | ||
| artifactLocation : ArtifactLocation | ||
| region : Region | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| /-- SARIF location wrapper -/ | ||
| structure SarifLocation where | ||
| physicalLocation : PhysicalLocation | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| /-- SARIF message -/ | ||
| structure Message where | ||
| text : String | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| /-- SARIF result level -/ | ||
| inductive Level where | ||
| | none -- Verification passed | ||
| | note -- Informational | ||
| | warning -- Unknown result or potential issue | ||
| | error -- Verification failed | ||
| deriving Repr, DecidableEq | ||
|
|
||
| instance : ToString Level where | ||
| toString | ||
| | .none => "none" | ||
| | .note => "note" | ||
| | .warning => "warning" | ||
| | .error => "error" | ||
|
|
||
| instance : ToJson Level where | ||
| toJson level := Json.str (toString level) | ||
|
|
||
| instance : FromJson Level where | ||
| fromJson? j := do | ||
| let s ← j.getStr? | ||
| match s with | ||
| | "none" => pure .none | ||
| | "note" => pure .note | ||
| | "warning" => pure .warning | ||
| | "error" => pure .error | ||
| | _ => throw s!"Invalid SARIF level: {s}" | ||
|
|
||
| /-- SARIF result representing a single verification result -/ | ||
| structure Result where | ||
| /-- Stable identifier of the rule that was evaluated to produce the result --/ | ||
| ruleId : String | ||
| level : Level | ||
| message : Message | ||
| locations : Array SarifLocation := #[] | ||
| deriving Repr, ToJson, FromJson, DecidableEq | ||
|
|
||
| instance : Inhabited Result where | ||
| default := { ruleId := "", level := .none, message := { text := "" } } | ||
|
|
||
| /-- SARIF tool driver information -/ | ||
| structure Driver where | ||
| /-- The exact command-line tool in Strata --/ | ||
| name : String | ||
tautschnig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| version : String := "0.1.0" | ||
| informationUri : String := "https://github.com/strata-org/Strata" | ||
| deriving Repr, ToJson, FromJson, Inhabited | ||
|
|
||
| /-- SARIF tool information -/ | ||
| structure Tool where | ||
| driver : Driver | ||
| deriving Repr, ToJson, FromJson, Inhabited | ||
|
|
||
| /-- SARIF run representing a single analysis run -/ | ||
| structure Run where | ||
| tool : Tool | ||
| results : Array Result | ||
| deriving Repr, ToJson, FromJson | ||
|
|
||
| instance : Inhabited Run where | ||
| default := { tool := default, results := #[] } | ||
|
|
||
| /-- Top-level SARIF document -/ | ||
| structure SarifDocument where | ||
| version : String := "2.1.0" | ||
| /-- Schema URI as specified by SARIF 2.1.0 -/ | ||
| schema : String := "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json" | ||
| runs : Array Run | ||
| deriving Repr, ToJson, FromJson | ||
|
|
||
| /-! ## Utility Functions -/ | ||
|
|
||
| /-- Convert a location to SARIF format -/ | ||
| def locationToSarif (loc : Location) : SarifLocation := | ||
| let artifactLocation : ArtifactLocation := { uri := loc.uri } | ||
| let region : Region := { startLine := loc.startLine, startColumn := loc.startColumn } | ||
| { physicalLocation := { artifactLocation, region } } | ||
|
|
||
| /-- Convert SARIF document to JSON string -/ | ||
| def toJsonString (sarif : SarifDocument) : String := | ||
| Json.compress (ToJson.toJson sarif) | ||
|
|
||
| /-- Convert SARIF document to pretty-printed JSON string -/ | ||
| def toPrettyJsonString (sarif : SarifDocument) : String := | ||
| Json.pretty (ToJson.toJson sarif) | ||
|
|
||
| end Strata.Sarif | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.