Open
Conversation
* Update bom to 26.74.0 (excluding spanner) * remove explicit sm dep
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.58.0 to 1.59.1. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](googleapis/google-cloud-go@spanner/v1.58.0...storage/v1.59.1) --- updated-dependencies: - dependency-name: cloud.google.com/go/storage dependency-version: 1.59.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This PR adds documentation and examples for using calcite_connection_properties in YAML SQL transforms. This enables users to use dialect-specific SQL functions such as PostgreSQL's INITCAP and STRING_TO_ARRAY functions. Changes: - Add new example file: sql/sql_calcite_connection_properties.yaml showing how to configure PostgreSQL functions in YAML SQL queries - Add test case in tests/sql.yaml to verify calcite_connection_properties works with PostgreSQL functions Fixes #36614
* Install yarn via corepack * Add corepack home dir
- Add filter() method following same pattern as map() and flatMap() - Add 5 test cases covering various filtering scenarios - Support optional context parameter for side inputs
* Fixed failures in Python PreCommit tests * Remove explicit runner parameter * Removed Python 3.13 special case for bundle timeout
* Simplify encoding kw_only args and test init=False args. 1. Replace the helper function for determining whether to use kw_only with a simpler to follow condition. 2. Add test coverage for the init=False case. * Fix pre-commit.
#36667) the change stream partition mode For MUTABLE_KEY_RANGE change stream, use read_proto_bytes_, else use read_json_
… for portable SchemaCoders.
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.21 to 0.0.22. - [Release notes](https://github.com/Kludex/python-multipart/releases) - [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md) - [Commits](Kludex/python-multipart@0.0.21...0.0.22) --- updated-dependencies: - dependency-name: python-multipart dependency-version: 0.0.22 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Enable some tests in runner v2 batch mode. * Skip the failed tests after re-enabling some categories.
… only claim timestamps that have been fully processed from the restriction tracker. (#37326)
* Update yaml colab notebook * add runners * remove old copy * update yaml version * change to master * fix error in patch and update yaml example * correct path
…#37439) Bumps [github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) from 2.12.3 to 2.12.4. - [Release notes](https://github.com/nats-io/nats-server/releases) - [Commits](nats-io/nats-server@v2.12.3...v2.12.4) --- updated-dependencies: - dependency-name: github.com/nats-io/nats-server/v2 dependency-version: 2.12.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add exception chaining to preserve error context - Add 'from e' to exception re-raises in CloudSQLEnrichmentHandler - Add exception chaining in processes.py for OSError and CalledProcessError - Improve logging in core.py to preserve traceback context This improves debuggability by preserving the full exception chain, following Python PEP 3134 best practices. Fixes #37422 * Fix yapf formatting for logging.warning statement * Fix yapf formatting: put logging arguments on single line
…e cache directory (#37360) * Fix cached wheels used in future runs * address review comments * run post tests * add .github/trigger_files/beam_PostCommit_Python_Examples_Dataflow.json
* Fix python postcommit * Trigger postcommit
* remove pubsublite from java sdk * revert builde file change impacting pubsub * remove unused dependencies
Add Apache Beam installation to Beam Playground Container Dockerfile
…hutdown instead of being retried. This matches MemoryMonitor detection of gc thrashing and prevents workers from remaining in broken state that may be fixed by restarting (#37749)
* add more parquet options * comments * more tests and use default
…nd up in the same shard as other records as it is random. Simplify the test to use Iterables instead of arrays.
#37510) * If the partition count or kafka IO size is large, then skip committing offsets that are not changed. Reduce kafka commit load * Address PR review feedback for idle partition optimization - Refactor commitCheckpointMark to use Java streams (per @johnjcasey) Changed from explicit for-loop to streams-based filtering for better code consistency with existing patterns - Add debug logging for idle partitions (per @tomstepp) Log the count of idle partitions skipped during each commit to aid in monitoring and debugging the optimization - Implement time-based periodic commits (per @tomstepp) Track last commit time per partition and ensure commits happen at least every 10 minutes even for idle partitions. This supports time lag monitoring use cases where customers track time since last commit. - Add unit test for idle partition behavior (per @tomstepp) New test KafkaUnboundedReaderIdlePartitionTest verifies that: * Idle partitions are not committed repeatedly * Active partitions trigger commits correctly * Uses mock consumer to track commit calls All changes maintain backward compatibility and follow Apache Beam coding standards (spotless formatting applied). * Fix test to follow Beam patterns for MockConsumer initialization Rewrote KafkaUnboundedReaderIdlePartitionTest to follow the exact pattern used in KafkaIOTest.java: - Proper MockConsumer initialization with partition metadata - Correct setup of beginning/end offsets - Consumer records with proper offsets and timestamps - schedulePollTask for record enqueueing based on position - Override commitSync to track commit calls - Use reader.start() before reader.advance() This ensures the test properly initializes the Kafka consumer and doesn't fail with IllegalStateException during source.split(). --------- Co-authored-by: Kishore Pola <kpola@paloaltonetworks.com>
…ency-1 Fix Python PostCommit Dependency
This message is printed on every import of apache-beam, which is unnecessarily verbose.
* Move FileIO close from RecordWriter to RecordWriterManager * fix * clarify FileIO ownership comments and verify close
…7760) * Fix BadImport ErrorProne violations across multiple modules * spotless
* ITs for RESTCatalog using BLMS * update rest catalog config * use top-level gcs bucket for warehouse
…t cannot be decoded successfully (#37762) Such messages will log an error but are otherwise discarded. Update PaneInfoCoder to throw a CoderException instead of ArrayOutOfBoundsException
Fix PreCommit Python ML tests with ML deps installed
Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.1.2 to 3.1.5. - [Changelog](https://github.com/isaacs/minimatch/blob/main/changelog.md) - [Commits](isaacs/minimatch@v3.1.2...v3.1.5) --- updated-dependencies: - dependency-name: minimatch dependency-version: 3.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )