Revisions of python-distributed
buildservice-autocommit
accepted
request 1127185
from
Ondřej Súkup (mimi_vx)
(revision 148)
baserev update by copy to link target
Ondřej Súkup (mimi_vx)
accepted
request 1127182
from
Ondřej Súkup (mimi_vx)
(revision 147)
- Update to 2023.11.0 - add distributed-ignore-rerun.patch to ignore pytest runners during test code introspection * Zero-copy P2P Array Rechunking * Deprecating PyArrow <14.0.1 * Improved PyArrow filesystem for Parquet * Improve Type Reconciliation in P2P Shuffling * official support for Python 3.12 * Reduced memory pressure for multi array reductions * improved P2P shuffling robustness * Reduced scheduler CPU load for large graphs
buildservice-autocommit
accepted
request 1110217
from
Dirk Mueller (dirkmueller)
(revision 146)
baserev update by copy to link target
Dirk Mueller (dirkmueller)
accepted
request 1110163
from
Benjamin Greiner (bnavigator)
(revision 145)
- Update to 2023.9.1 ## Enhancements Raise in P2P if column dtype is wrong (GH#8167) Hendrik Makait * Auto-fail tasks with deps larger than the worker memory (GH#8135) crusaderky * Make workers table sortable (GH#8153) Jacob Tomlinson * Support for unsetting environment variables (GH#8144) crusaderky ## Deprecations * Deprecate asynchronous Listener.stop() (GH#8151) Hendrik Makait - Release 2023.9.0 ## Enhancements Don’t capture functools frames in code (GH#8146) crusaderky * Ignore asyncio when scraping code (GH#8122) Matthew Rocklin * Remove stringification (GH#8083) Florian Jetter ## Bug Fixes * Ensure NannyPlugin are always installed (GH#8107) Florian Jetter * Don’t use exception hooks to shorten tracebacks (GH#8127) crusaderky * Fix P2P shuffle with LocalCluster(..., processes=False) (GH#8125) Hendrik Makait * Merge with P2P shuffle fails if left_index or right_index is True (GH#8121) Patrick Hoefler - Release 2023.8.1 ## New Features * Add memray integration (GH#8044) Florian Jetter ## Enhancements * Await async listener.stop in Worker.close (GH#8118) Hendrik Makait * Minor fixes in memray (GH#8113) Florian Jetter * Enable basic p2p shuffle for dask-cudf (GH#7743) Richard (Rick) Zamora * Don’t shut down unresponsive workers on gather() (GH#8101) crusaderky * Propagate CancelledError in gather_from_workers (GH#8089) crusaderky * Better logging for anomalous task termination (GH#8082) crusaderky ## Bug Fixes * Handle null partitions in P2P shuffling (GH#8116) Hendrik Makait
buildservice-autocommit
accepted
request 1090796
from
Steve Kowalik (StevenK)
(revision 144)
baserev update by copy to link target
Steve Kowalik (StevenK)
committed
(revision 143)
- Ratchet up bokeh to >= 3.1, since upstream supports it. - Also BuildRequire sparse for Python 3.11.
Steve Kowalik (StevenK)
accepted
request 1090160
from
Benjamin Greiner (bnavigator)
(revision 142)
- Update to 2023.5.1 * This release drops support for Python 3.8 ## Enhancements * Exclude IPython code from computations (GH#7788) Miles * Drop Python 3.8 support (GH#7840) Thomas Grainger * Add storage_options to performance_report (GH#7636) ypogorelova * Don’t warn about mismatched msgpack (GH#7839) Irina Truong * Clean up sys.path on Server shutdown (GH#7838) James Bourbeau * Dashboard: Fine Performance Metrics (GH#7725) Miles ## Bug Fixes * Properly handle unknown chunk sizes in P2P rechunking (GH#7856) Hendrik Makait * Minimal change to work around (GH#7726) / support for UCX (GH#7851) Benjamin Zaitlen * Don’t end computations until cluster is truly idle (GH#7790) crusaderky - Release 2023.5.0 ## Enhancements * Client.upload_file send to both Workers and Scheduler and rename scratch directory (GH#7802) Miles * Allow dashboard to be used with bokeh prereleases (GH#7814) James Bourbeau ## Bug Fixes * Ensure log_event of non-msgpack serializable object do not kill servers (GH#7472) Florian Jetter * Fix test_nanny.py duplicated pytestmark definitions (GH#7819) Thomas Grainger * Fix flaky test_dask_worker.py::test_single_executable_deprecated (GH#7817) Thomas Grainger
buildservice-autocommit
accepted
request 1077429
from
Steve Kowalik (StevenK)
(revision 141)
baserev update by copy to link target
Steve Kowalik (StevenK)
accepted
request 1077366
from
Benjamin Greiner (bnavigator)
(revision 140)
- Drop python38 test flavor
buildservice-autocommit
accepted
request 1074795
from
Matej Cepl (mcepl)
(revision 139)
baserev update by copy to link target
Matej Cepl (mcepl)
accepted
request 1074755
from
Benjamin Greiner (bnavigator)
(revision 138)
- Update to 2023.3.2 ## Enhancements * Enhanced thread-safety in zict.File (GH#7691) crusaderky * Future deserialization without available client (GH#7580) Florian Jetter * Support adjusting GIL monitoring interval (GH#7650) Miles * Gracefully stop GIL monitoring if running (GH#7652) Miles * Fine performance metrics for execute, gather_dep, etc. (GH#7586) crusaderky * Add GIL metric to dashboard (GH#7646) Miles * Expose scheduler idle via RPC and HTTP API (GH#7642) Jacob Tomlinson * Add full dashboard link in scheduler logs (GH#7631) Miles ## Bug Fixes * Tell workers when their peers have left (so they don’t hang fetching data from them) (GH#7574) Thomas Grainger * Fix regression in dashboard after (GH#7586) (GH#7683) crusaderky * Fix OverflowError in Cluster._sync_cluster_info() (GH#7648) Hendrik Makait * Ensure that serialized data is measured correctly (GH#7593) Florian Jetter ## Maintenance * Avoid bool object has no attribute close in @gen_cluster (GH#7657) Thomas Grainger * Fix failing test_server_close_stops_gil_monitoring (GH#7659) James Bourbeau * Remove weakref finalizer for Offload Executor (GH#7644) Florian Jetter - Enable python311 build and test: numba not a strict requirement
buildservice-autocommit
accepted
request 1071516
from
Matej Cepl (mcepl)
(revision 137)
baserev update by copy to link target
Matej Cepl (mcepl)
accepted
request 1071308
from
Benjamin Greiner (bnavigator)
(revision 136)
- Skip more flaky tests - Update to 2023.3.1 ## Enhancements * Add Jupyter link to dashboard menu if --jupyter flag is set (GH#7638) Jacob Tomlinson * Bump minimum click version from 7.0 to 8.0 (GH#7637) Miles * Extend dask metapackage dependencies (GH#7630) James Bourbeau * Further improvements to Client.restart_workers (GH#7620) Miles * P2P offload get_output_partition (GH#7587) Florian Jetter * Initial integration of GIL contention metric (GH#7624) Miles * Add dashboard documentation links (GH#7610) Miles * Rename shuffle/rechunk config option/kwarg to method (GH#7623) Hendrik Makait * Return results in restart_workers (GH#7606) Miles * Ensure client key cancellation uses ordered messages (GH#7583) Florian Jetter ## Bug Fixes * Fix undefined async_wait_for -> async_poll_for (GH#7627) Miles * Don’t send client heartbeat without a scheduler_comm (GH#7612) James Bourbeau * Do not unspill on free-keys (GH#7607) crusaderky ## Documentation * Add notes to Client.submit, Client.map, and Client.scatter with the description of the current task graph resolution algorithm limitations (GH#7588) Eugene Druzhynin ## Maintenance * Use range with pickle protocol versions (GH#7635) jakirkham * Share thread pool among P2P shuffle runs (GH#7621) Hendrik Makait
buildservice-autocommit
accepted
request 1070124
from
Dirk Mueller (dirkmueller)
(revision 135)
baserev update by copy to link target
Dirk Mueller (dirkmueller)
committed
(revision 134)
- update to 2023.3.0: * Remove pyarrow dependency for rechunking * Update rechunk_transfer and rechunk_unpack errors * Remove dead code and document arguments to ShardBuffer constructors * Fix tests for p2p by default * Remove obsolete cast
buildservice-autocommit
accepted
request 1068133
from
Matej Cepl (mcepl)
(revision 133)
baserev update by copy to link target
Matej Cepl (mcepl)
accepted
request 1068088
from
Benjamin Greiner (bnavigator)
(revision 132)
- Prepare test multiflavors for python311, but skip python311 * Numba is not ready for python 3.11 yet gh#numba/numba#8304 - Update to 2023.2.0 ## Enhancements * Rate limit the worker memory logs (GH#7529) Florian Jetter * Move P2P barrier logic to scheduler extension (GH#7519) Hendrik Makait
buildservice-autocommit
accepted
request 1063411
from
Daniel Garcia (dgarcia)
(revision 131)
baserev update by copy to link target
Daniel Garcia (dgarcia)
accepted
request 1063268
from
Benjamin Greiner (bnavigator)
(revision 130)
- Update to 2023.1.1 ## Enhancements * P2P shuffle deduplicates data and can be run several times (GH#7486) Hendrik Makait * Reverse order of get_logs() and get_worker_logs() (GH#7475) Nicholas R. Knezek * Add prometheus metric for time and memory used per task prefix (GH#7406) Thomas Grainger * Additive worker counts in Prometheus (GH#7468) crusaderky * Add help tool for taskstream (GH#7478) Florian Jetter ## Bug Fixes * Do not allow for a worker to reject a drop replica request (GH#7490) Hendrik Makait * Fix un/packing for namedtuples with custom constructors (GH#7465) antonymayi * Documentation * Remove timeout= from docstring example for worker_client (GH#7497) Florian Jetter ## Maintenance * Ignore get_default_shuffle_algorithm linting issue (GH#7506) Florian Jetter * Remove set_config when using default client (GH#7482) Florian Jetter * Update gpuCI RAPIDS_VER to 23.04 (GH#7501) * Fix test_balance_expensive_tasks and improve helper functions in test_steal.py (GH#7253) Hendrik Makait * Sign every compute task with run ID to correlate response (GH#7463) Hendrik Makait - Release 2023.1.0 ## New Features * Add local SubprocessCluster that runs workers in separate processes (GH#7431) Hendrik Makait ## Enhancements * Ensure client session is quiet after cluster.close() or client.shutdown() (GH#7429) James Bourbeau * Set lifetime-stagger default value to None (GH#7445) bstadlbauer * Memory thresholds should never be exactly 0.0 (GH#7458) Stuart Berg * Remove the Incorrect-Sizeof-Warning (GH#7450) Mads R. B. Kristensen * Log exceptions in P2P shuffle tasks (GH#7442) Hendrik Makait * Add support for packing namedtuple and add test for future resolution in submit (GH#7292) Andrew * Avoid deep copy on lz4 decompression (GH#7437) crusaderky * Avoid deep copy of numpy buffers on unspill (GH#7435) crusaderky * Don’t error when clicking on empty task stream plot (GH#7432) James Bourbeau * Do not count spilled memory when comparing vs. process memory (GH#7430) crusaderky * Stop Client periodic callbacks during shutdown() (GH#7428) James Bourbeau * Add dask spec CLI (GH#7427) Matthew Rocklin * Create new zstd (de)compressor for each compression call (GH#7404) Dylan Wragge * Rename managed_in_memory etc. to match GUI (GH#7418) crusaderky * Warn users when sizeof() returns inflated output (GH#7419) crusaderky ## Bug Fixes * Ensure dicts are properly recognized as msgpack serializable (GH#7473) Florian Jetter * Reset state of ShuffleSchedulerExtension on restart (GH#7446) Hendrik Makait * Reject non-string column names in P2P shuffle (GH#7447) Hendrik Makait * Avoid int32 in dashboard (GH#7443) Matthew Rocklin * Fix P2PShuffle serialization for categorical data (GH#7410) Hendrik Makait * WorkerPorcess blocks on kill if still starting (GH#7424) Matthew Rocklin ## Documentation * Move Prometheus docs from dask/dask (GH#7405) crusaderky ## Maintenance * Various cleanups in semaphore (GH#5885) Florian Jetter * test_rlimit fails on MacOSX (GH#7457) crusaderky * Bump actions/checkout from 3.2.0 to 3.3.0 (GH#7464) * Remove conditional imports of psutil (GH#7462) crusaderky * Drop support for zict < 2.1.0 (GH#7456) crusaderky * Fix flaky test_digests (GH#7454) crusaderky * Add minimum dependency testing to CI (GH#7285) Charles Blackmon-Luca * Avoid overflow in statitics.mean (GH#7426) Matthew Rocklin * Ignore numpy bool8 deprecation (GH#7423) Matthew Rocklin * Add missing skips for pyarrow (GH#7416) Elliott Sales de Andrade * Be more permissive about expected ciphers in tests (GH#7417) Elliott Sales de Andrade * Revert “TST: Fetch executables from build root (GH#2551)” (GH#7415) Elliott Sales de Andrade
buildservice-autocommit
accepted
request 1056244
from
Matej Cepl (mcepl)
(revision 129)
baserev update by copy to link target
Displaying revisions 21 - 40 of 168