-
Notifications
You must be signed in to change notification settings - Fork 3.2k
feat(tinybird): added tinybird block #2781
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: staging
Are you sure you want to change the base?
Conversation
… fixes, subflow resize clamping
…ribe, auth checks, new db indexes
…dioai#2481) The realtime service network policy was missing the custom egress rules section that allows configuration of additional egress rules via values.yaml. This caused the realtime pods to be unable to connect to external databases (e.g., PostgreSQL on port 5432) when using external database configurations. The app network policy already had this section, but the realtime network policy was missing it, creating an inconsistency and preventing the realtime service from accessing external databases configured via networkPolicy.egress values. This fix adds the same custom egress rules template section to the realtime network policy, matching the app network policy behavior and allowing users to configure database connectivity via values.yaml.
…rovements, additional kb tag types
…ts, LLM providers standardization
…, jira service management tools
…ext menu (simstudioai#2672) * feat(logs-context-menu): consolidated logs utils and types, added logs record context menu (simstudioai#2659) * feat(email): welcome email; improvement(emails): ui/ux (simstudioai#2658) * feat(email): welcome email; improvement(emails): ui/ux * improvement(emails): links, accounts, preview * refactor(emails): file structure and wrapper components * added envvar for personal emails sent, added isHosted gate * fixed failing tests, added env mock * fix: removed comment --------- Co-authored-by: waleed <[email protected]> * fix(logging): hitl + trigger dev crash protection (simstudioai#2664) * hitl gaps * deal with trigger worker crashes * cleanup import strcuture * feat(imap): added support for imap trigger (simstudioai#2663) * feat(tools): added support for imap trigger * feat(imap): added parity, tested * ack PR comments * final cleanup * feat(i18n): update translations (simstudioai#2665) Co-authored-by: waleedlatif1 <[email protected]> * fix(grain): updated grain trigger to auto-establish trigger (simstudioai#2666) Co-authored-by: aadamgough <[email protected]> * feat(admin): routes to manage deployments (simstudioai#2667) * feat(admin): routes to manage deployments * fix naming fo deployed by * feat(time-picker): added timepicker emcn component, added to playground, added searchable prop for dropdown, added more timezones for schedule, updated license and notice date (simstudioai#2668) * feat(time-picker): added timepicker emcn component, added to playground, added searchable prop for dropdown, added more timezones for schedule, updated license and notice date * removed unused params, cleaned up redundant utils * improvement(invite): aligned styling (simstudioai#2669) * improvement(invite): aligned with rest of app * fix(invite): error handling * fix: addressed comments --------- Co-authored-by: Emir Karabeg <[email protected]> Co-authored-by: Vikhyath Mondreti <[email protected]> Co-authored-by: waleedlatif1 <[email protected]> Co-authored-by: Adam Gough <[email protected]> Co-authored-by: aadamgough <[email protected]>
…in integration updates
…ion, autoconnect improvements, billing fixes
…ow preview enhancements
|
@lakeesiv is attempting to deploy a commit to the Sim Team on Vercel. A member of the Team first needs to authorize it. |
Greptile OverviewGreptile SummaryThis PR adds Tinybird integration to Sim with two operations: Send Events and Query. Implementation OverviewThe implementation follows the established block/tool pattern used throughout the codebase:
ArchitectureThe block uses a dropdown to switch between operations, with conditional fields that show/hide based on the selected operation. The Strengths✅ Follows established patterns from similar blocks (PostHog, Sentry) Issues FoundAll issues are style/improvement suggestions - no blocking bugs:
Testing NotesThe PR indicates tests were not added. Reviewers should verify:
Confidence Score: 4/5
Important Files ChangedFile Analysis
Sequence DiagramsequenceDiagram
participant User
participant TinybirdBlock
participant ToolRegistry
participant EventsTool
participant QueryTool
participant TinybirdAPI
Note over User,TinybirdAPI: Send Events Operation
User->>TinybirdBlock: Configure Send Events operation
User->>TinybirdBlock: Provide datasource, data, token
TinybirdBlock->>TinybirdBlock: Parse params & convert wait boolean
TinybirdBlock->>ToolRegistry: Route to tinybird_events tool
ToolRegistry->>EventsTool: Execute with params
EventsTool->>EventsTool: Build URL with query params
EventsTool->>EventsTool: Apply gzip compression (if enabled)
EventsTool->>TinybirdAPI: POST /v0/events
TinybirdAPI-->>EventsTool: Response with successful_rows, quarantined_rows
EventsTool->>EventsTool: Transform response
EventsTool-->>User: Return output
Note over User,TinybirdAPI: Query Operation
User->>TinybirdBlock: Configure Query operation
User->>TinybirdBlock: Provide SQL query, token
TinybirdBlock->>TinybirdBlock: Parse params
TinybirdBlock->>ToolRegistry: Route to tinybird_query tool
ToolRegistry->>QueryTool: Execute with params
QueryTool->>QueryTool: Build form-encoded body
QueryTool->>TinybirdAPI: POST /v0/sql
TinybirdAPI-->>QueryTool: Response (JSON/CSV/TSV)
QueryTool->>QueryTool: Detect format via content-type
QueryTool->>QueryTool: Parse JSON or return raw text
QueryTool-->>User: Return data, rows, statistics
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
4 files reviewed, 4 comments
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…o lakees/tinybird
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
6 files reviewed, 6 comments
| output: { | ||
| data: data.data || [], | ||
| rows: data.rows || 0, | ||
| statistics: data.statistics | ||
| ? { | ||
| elapsed: data.statistics.elapsed, | ||
| rows_read: data.statistics.rows_read, | ||
| bytes_read: data.statistics.bytes_read, | ||
| } | ||
| : undefined, | ||
| }, | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The JSON response handling assumes a specific structure (data.data, data.rows, data.statistics), but doesn't validate that these fields exist or have the expected types before accessing nested properties.
If Tinybird's API changes or returns a different structure, this could throw unexpected errors. Consider adding validation:
| output: { | |
| data: data.data || [], | |
| rows: data.rows || 0, | |
| statistics: data.statistics | |
| ? { | |
| elapsed: data.statistics.elapsed, | |
| rows_read: data.statistics.rows_read, | |
| bytes_read: data.statistics.bytes_read, | |
| } | |
| : undefined, | |
| }, | |
| } | |
| return { | |
| success: true, | |
| output: { | |
| data: Array.isArray(data.data) ? data.data : [], | |
| rows: typeof data.rows === 'number' ? data.rows : 0, | |
| statistics: data.statistics && typeof data.statistics === 'object' | |
| ? { | |
| elapsed: data.statistics.elapsed ?? 0, | |
| rows_read: data.statistics.rows_read ?? 0, | |
| bytes_read: data.statistics.bytes_read ?? 0, | |
| } | |
| : undefined, | |
| }, | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/sim/tools/tinybird/query.ts
Line: 104:115
Comment:
The JSON response handling assumes a specific structure (`data.data`, `data.rows`, `data.statistics`), but doesn't validate that these fields exist or have the expected types before accessing nested properties.
If Tinybird's API changes or returns a different structure, this could throw unexpected errors. Consider adding validation:
```suggestion
return {
success: true,
output: {
data: Array.isArray(data.data) ? data.data : [],
rows: typeof data.rows === 'number' ? data.rows : 0,
statistics: data.statistics && typeof data.statistics === 'object'
? {
elapsed: data.statistics.elapsed ?? 0,
rows_read: data.statistics.rows_read ?? 0,
bytes_read: data.statistics.bytes_read ?? 0,
}
: undefined,
},
}
```
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If tinybirds api changes an error is better than whatever its suggested here
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…o lakees/tinybird
| const responseText = await response.text() | ||
| const contentType = response.headers.get('content-type') || '' | ||
|
|
||
| if (!response.ok) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lakeesiv one thing: if there is an error, it won't go into the transformResponse codepath and will go through the consolidated error service in the tools/ directory, I believe in executeTool, so we don't necessarily need error handling here and can solely rely on the error handling there. if tinybird throws a unqiue error type that is not already handled in the error extractor, then we can add the pattern there to make sure it actually extracts the error message. this goes for the events as well. besides that, LGTM
Summary
Adds Tinybird integration with two operations:
Send Events: Stream JSON/NDJSON events to Tinybird Data Sources with compression and acknowledgment options
Query: Execute SQL queries against Tinybird Pipes and Data Sources with support for multiple output formats (JSON, CSV, TSV, etc.)
Includes block configuration, tool implementations, API routes, and a custom icon component.
Type of Change
Testing
Tested manually
Checklist