Changelog

Upcoming Release

Version 0.8.0 (August 28, 2019)

Version 0.7.4 (June 28, 2019)

  • Close gRPC channels automatically on client shutdown (PR #179)

  • Silence client shutdown logging that was confusing to new users

Version 0.7.3 (May 6, 2019)

  • Add ApplicationClient.add_container, providing the ability to add a container to a service with optional service configuration overrides (PR #174)

Version 0.7.2 (May 2, 2019)

  • Fix bug in shutdown logic preventing shutdown on certain container failures (Issue #172, PR #173)

Version 0.7.1 (April 25, 2019)

Version 0.7.0 (April 22, 2019)

  • Add Client.get_nodes for querying status of YARN cluster nodes (Issue #155, PR #156)

  • Expose information on YARN queues through Client.get_queue, Client.get_child_queues, and Client.get_all_queues (PR #159)

  • Improve error in build script if maven isn’t installed (Issue #158, PR #160)

  • Add support for moving applications between queues (PR #161)

  • Support more filters in Client.get_applications (Issue #133, PR #162)

  • Log skein version in both Driver and ApplicationMaster Java processes (PR #163)

  • Enable testing on Hadoop 3/CDH 6 (Issue #153, PR #164)

  • Explicitly use ipv4 for Python <-> Java connection, removing potential for each binding to different interfaces (Issue #165, PR #166)

Version 0.6.1 (April 3, 2019)

  • Add support to ApplicationClient.scale for scaling by a delta in instances rather than a total instance count (PR #150)

  • Fix bug in finding container directory (PR #151)

Version 0.6.0 (March 21, 2019)

  • More robustly handle starting/stopping the global driver in the case of previous driver failure (PR #141, Issue #140)

  • Add allow_failures field to services (PR #145)

  • Better error messages for improperly specified files (PR #146, Issue #139)

  • Expose the absolute path to the current container working directory as skein.properties.container_dir (PR #147, Issue #138)

  • Prevent a race condition when creating new global credentials (PR #148, Issue #144)

  • Remove functionality previously deprecated in version 0.5.0 (PR #149)

Version 0.5.1 (January 21, 2019)

  • Fix bug preventing launching containers with vcores > 1 on some YARN configurations (PR #131)

  • Ensure application staging directory is cleaned up if submission fails (PR #132)

  • Refactored error handling in application master, improving error logs during application master failure (PR #134)

  • Decrease heartbeat interval during steady-state operation, reducing communication load on the resource manager (PR #137, Issue #135)

Version 0.5.0 (January 7, 2019)

  • Support login via keytab, allowing for long-running services (PR #115, Issue #103)

  • Rename daemon to driver everywhere, deprecating old methods/classes (PR #116)

  • Allow forwarding java options to the Skein driver (PR #117)

  • Fix bugs preventing the Skein driver from running inside a YARN container (PR #119)

  • Add support for running a single user process on the same node as the Application Master, allowing for faster application startup for single-container services (PR #120, Issue #118)

  • Ensure application directory is cleaned up, even during application master failure or if killed by other tools (PR #122)

  • Fix support for application retries (PR #122)

  • Deprecate commands field in favor of script (PR #125, Issue #121)

  • Add --force option to skein driver stop (PR #126, Issue #124)

  • Update Web UI to display new features (progress reports, application master/driver logs, etc…) (PR #127, PR #95, Issue #123)

  • Obtain a resource manager delegation token for all applications. Allows for applications to submit additional applications as needed (PR #127)

Version 0.4.1 (December 7, 2018)

  • Reduce size of Skein’s JAR by selectively culling unnecessary resources (PR #109)

  • Use Protobuf Lite to further reduce JAR size (PR #111)

  • Normalize application specification consistently between Client.submit and Client.submit_and_connect (PR #114, Issue #110).

Version 0.4.0 (December 5, 2018)

  • Add support for proxying user credentials, allowing submitted applications to run as a different user than the submitter (PR #101)

  • Support running on MapR provided clusters (PR #105)

  • Allow TLS credentials to be configured per-application (PR #107)

  • Silence extraneous gRPC logged warning (“Fork support only compatible with epoll1 and poll polling strategies”).This warning doesn’t apply to our use case, and will not be raised in a future release of gRPC. (PR #107)

  • Upgrade gRPC version to 1.16 (PR #107)

  • Silence deprecation warnings in Python 3.7 (PR #108)

Version 0.3.1 (October 29, 2018)

  • Use NM_HOST to determine hostname, fixes connection issues on systems with alternate network interfaces (e.g. infiniband) (PR #97)

  • Fix accidental reliance on JAVA_HOME being defined (PR #100)

Version 0.3.0 (October 26, 2018)

  • Add support for YARN node label expressions (PR #44)

  • Allow memory requirements to be specified with human-readable units (PR #87, Issue #86)

  • Add support for YARN node and rack requirements/suggestions (PR #90, Issue #89)

  • Allow setting diagnostics message on user-requested shutdown (PR #92)

  • Add ability to set application progress (PR #93, Issue #88)

  • Error nicely if user forgets to kinit (PR #94)

  • Improve logging messages in client daemon (PR #95)

  • Support configurable logging for client daemon (PR #96)

Version 0.2.0 (October 11, 2018)

  • Add support for specifying additional filesystems, needed for ViewFs (PR #58)

  • Add a Web UI for viewing application status and logs (PR #68, PR #42, Issue #34)

  • Cleanup staging directory for killed applications (PR #71, Issue #69)

  • Support application-level Access Control Lists (ACLs) (PR #78, Issue #74)

  • Add support for user-defined pages in the Web UI (PR #72)

  • Remove unneeded resources to reduce JAR size (PR #83)

  • Support custom logging configurations (PR #84, Issue #79)

  • Improve logging messages and granularity (PR #84, Issue #79)

  • Add exit_message field for completed containers to aid in debugging (PR #84, PR #66)

Version 0.1.1 (August 6, 2018)

  • Fix bug in key ownership model (PR #53)

  • Cleanup and document skein.recipes (PR #50)

Version 0.1.0 (August 1, 2018)

  • Initial Public Release

  • Clean rewrite of Key-Value Store to support more powerful operations (PR #40, Issue #33, Issue #35)

  • Expose container node addresses in Python API (PR #39, Issue #38)

  • Expose application and container ids, as well as resource limits in running containers via environment variables (PR #37, Issue #32).

  • Cleanup internals and rework public API (PR #36)

  • Correctly handle multiple values in LOCAL_DIRS (PR #31)

Version 0.0.4 (July 3, 2018)

  • Initial Alpha Release