File python-elasticsearch-curator.changes of Package python-elasticsearch-curator
-------------------------------------------------------------------
Thu Jan 10 13:26:04 UTC 2019 - Thomas Bechtold <tbechtold@suse.com>
- update to 5.5.2:
* The ``alias``, ``restore``, ``rollover``, and ``shrink`` actions have been
added to ``curator_cli``, along with a revamped method to manage/add
actions in the future.
* Updated ``certifi`` dependency to ``2018.4.16``
* Added ``six`` dependency
* Permit the use of versions 6.1 and greater of the ``elasticsearch`` python
module. There are issues with SSL contexts in the 6.0 release that prevent
Curator from being able to use this version. Currently the requirement
version string is ``elasticsearch>=5.5.2,!=6.0.0,<7.0.0``
* Start of pylint cleanup, and use of `six` `string_types`. (untergeek)
* `unit_count_pattern` setting can cause indices to mistakenly be included
in an index filter. Fixed in #1206 (soenkeliebau)
* Fix rollover _check_max_size() call. Reported in #1202 by @diranged
(untergeek).
* Update tested versions of Elasticsearch. (untergeek).
* Update setup.cfg to install dependencies during source install. (untergeek)
* Fix reference to unset variable name in log output at
https://github.com/elastic/curator/blob/v5.5.1/curator/actions.py#L2145
It should be `idx` instead of `index`. (untergeek).
* Alias action should raise `NoIndices` exception if `warn_if_no_indices` is
`True`, and no `add` or `remove` sub-actions are found, rather than raising
an `ActionError`. Reported in #1209 (untergeek).
* Clarify inclusive filtering for allocated filter. Fixed in #1203 (geekpete)
* Fix Kibana filter description. #1199 (quartett-opa)
* Add missing documentation about the ``new_name`` option for rollover.
Reported in #1197 (untergeek)
* Fix ``pip`` installation issues for older versions of Python
#1183 (untergeek)
* Add ``wait_for_rebalance`` as an option for ``shrink`` action. By default
the behavior remains unchanged. You can now set this to False though to
allow the shrink action to only check that the index being shrunk has
finished being relocated and it will not wait for the cluster to
rebalance. #1129 (tschroeder-zendesk)
* Work around for extremely large cluster states. #1142 (rewiko)
* Add CI tests for Elasticsearch versions 6.1 and 6.2 (untergeek)
* Add Elasticsearch datemath support for snapshot names #1078 (untergeek)
* Support ``max_size`` as a rollover condition for Elasticsearch versions
6.1.0 and up. #1140 (untergeek)
* Skip indices with a document count of 0 when using ``source: field_stats``
to do ``age`` or ``period`` type filtering. #1130 (untergeek)
* Fix missing node information in log line. #1142 (untergeek)
* Fix default options in code that were causing schema validation errors
after ``voluptuous`` upgrade to 0.11.1.
Reported in #1149, fixed in #1156 (untergeek)
* Disallow empty lists as reindex source. Raise exception if that happens.
Reported in #1139 (untergeek)
* Set a ``timeout_override`` for ``delete_snapshots`` to catch cases where
slower repository network and/or disk access can cause a snapshot delete
to take longer than the default 30 second client timeout. #1133 (untergeek)
* Add AWS ES 5.1 support. #1172 (wanix)
* Add missing ``period`` filter arguments for ``delete_snapshots``.
Reported in #1173 (untergeek)
* Fix kibana filtertype to catch newer index names.
Reported in #1171 (untergeek)
* Re-order the closed indices filter for the Replicas action to take place
`before` the empty list check.
Reported in #1180 by ``@agomerz`` (untergeek)
* Deprecate testing for Python 3.4. It is no longer being supported by
Python.
* Increase logging to show error when ``master_only`` is true and there are
multiple hosts.
* Correct a misunderstanding about the nature of rollover conditions.
#1144 (untergeek)
* Correct links to the field_stats API, as it is non-existent in
Elasticsearch 6.x. (untergeek)
* Add a warning about using forcemerge on active indices.
#1153 (untergeek)
* Fix select URLs in pip installation from source to not be 404
#1133 (untergeek)
* Fix an error in regex filter documentation #1138 (arne-cl)
* Improve Dockerfile to build from source and produce slimmer image
#1111 (mikn)
* Fix ``filter_kibana`` to correctly use ``exclude`` argument
#1116 (cjuroz)
* Fix `ssl_no_validate` behavior within AWS ES #1118 (igalarzab)
* Improve command-line exception management #1119 (4383)
* Make ``alias`` action always process ``remove`` before ``add``
to prevent undesired alias removals. #1120 (untergeek)
* Bump ES versions in Travis CI
* Remove ``unit_count`` parameter doc for parameter that no longer
exists #1107 (dashford)
* Add missing ``exclude: True`` in ``timestring`` docs #1117 (GregMefford)
* Support for Elasticsearch 6.0!!! Yes!
* The ``field_stats`` API may be gone from Elasticsearch, but its utility
cannot be denied. And so, Curator has replaced the ``field_stats`` API
call with a small aggregation query. This will be perhaps a bit more
costly in performance terms, as this small aggregation query must be made
to each index in sequence, rather than as a one-shot call, like the
``field_stats`` API call. But the benefit will remain available, and
it's the only major API that did not persevere between Elasticsearch 5.x
and 6.x that was needed by Curator.
* With the period filter and field_stats, it is useful to match indices
that fit `within` the period, rather than just their start dates. This
is now possible with ``intersect``. See more in the documentation.
Requested in #1045. (untergeek)
* Add a ``restore`` function to ``curator_cli`` singleton. Mentioned in
#851 (alexef)
* Add ``pattern`` to the ``count`` filter. This is particularly useful
when working with rollover indices. Requested in #1044 (untergeek)
* The ``es_repo_mgr create`` command now can take ``skip_repo_fs_check`` as
an argument (default is False) #1072 (alexef)
* Add ``pattern_type`` feature expansion to the ``period`` filter. The
default behavior is ``pattern_type='relative'``, which preserves existing
behaviors so users with existing configurations can continue to use them
without interruption. The new ``pattern_type`` is ``absolute``, which
allows you to specify hard dates for ``date_from`` and ``date_to``, while
``date_from_format`` and ``date_to_format`` are strftime strings to
interpret the from and to dates. Requested in #1047 (untergeek)
* Add ``copy_aliases`` option to the ``shrink`` action. So this option is
only set in the ``shrink`` action. The default value of the option is
``copy_aliases: 'False'`` and it does nothing. If you set to
``copy_aliases: 'True'``, you could copy the aliases from the source index
to the target index. Requested in #1060 (monkey3199)
* IAM Credentials can now be retrieved from the environment using the Boto3
Credentials provider. #1084 (kobuskc)
* Delete the target index (if it exists) in the event that a shrink fails.
Requested in #1058 (untergeek)
* Fixed an integration test that could fail in the waning days of a month.
* Fix build system anomalies for both unix and windows.
* Set repository access to be https by default.
* Add documentation for ``copy_aliases`` option.
* Shrink action! Apologies to all who have patiently waited for this
feature. It's been a long time coming, but it is hopefully worth the
wait. There are a lot of checks and tests associated with this action,
as there are many conditions that have to be met in order for a shrink
to take place. Curator will try its best to ensure that all of these
conditions are met so you can comfortably rest assured that shrink will
work properly unattended. See the documentation for more information.
* The ``cli`` function has been split into ``cli`` and ``run`` functions.
The behavior of ``cli`` will be indistinguishable from previous releases,
preserving API integrity. The new ``run`` function allows lambda and other
users to `run` Curator from the API with only a client configuration file
and action file as arguments. Requested in #1031 (untergeek)
* Allow use of time/date string interpolation for Rollover index naming.
Added in #1010 (tschroeder-zendesk)
* New ``unit_count_pattern`` allows you to derive the ``unit_count`` from
the index name itself. This involves regular expressions, so be sure to
do lots of testing in ``--dry-run`` mode before deploying to production.
Added by (soenkeliebau) in #997
* Reindex ``request_body`` allows for 2 different ``size`` options. One
limits the number of documents reindexed. The other is for batch sizing.
The batch sizing option was missing from the schema validator. This has
been corrected. Reported in #1038 (untergeek)
* A few sundry logging and notification changes were made.
* An update to Elasticsearch 5.5.0 changes the behavior of
``filter_by_aliases``, differing from previous 5.x versions.
If a list of aliases is provided, indices must appear in `all` listed
aliases or a 404 error will result, leading to no indices being matched.
In older versions, if the index was associated with even one of the
aliases in aliases, it would result in a match.
Tests and documentation have been updated to address these changes.
* Debian 9 changed SSL versions, which means that the pre-built debian
packages no longer work in Debian 9. In the short term, this requires
a new repository. In the long term, I will try to get a better
repository system working for these so they all work together, better.
Requested in #998 (untergeek)
* Support date math in reindex operations better. It did work previously,
but would report failure because the test was looking for the index with
that name from a list of indices, rather than letting Elasticsearch do
the date math. Reported by DPattee in #1008 (untergeek)
* Under rare circumstances, snapshot delete (or create) actions could fail,
even when there were no snapshots in state ``IN_PROGRESS``. This was
tracked down by JD557 as a collision with a previously deleted snapshot
that hadn't finished deleting. It could be seen in the tasks API. An
additional test for snapshot activity in the tasks API has been added to
cover this scenario. Reported in #999 (untergeek)
* The ``restore_check`` function did not work properly with wildcard index
patterns. This has been rectified, and an integration test added to
satisfy this. Reported in #989 (untergeek)
* Make Curator report the Curator version, and not just reiterate the
elasticsearch version when reporting version incompatibilities. Reported
in #992. (untergeek)
* Fix repository/snapshot name logging issue. #1005 (jpcarey)
* Fix Windows build issue #1014 (untergeek)
* Fix/improve rST API documentation.
* Thanks to many users who not only found and reported documentation issues,
but also submitted corrections.
* Mock and cx_Freeze don't play well together. Packages weren't working, so
I reverted the string-based comparison as before.
* Index Settings are here! First requested as far back as #160, it's been
requested in various forms culminating in #656. The official documentation
addresses the usage. (untergeek)
* Remote reindex now adds the ability to migrate from one cluster to another,
preserving the index names, or optionally adding a prefix and/or a suffix.
The official documentation shows you how. (untergeek)
* Added support for naming rollover indices. #970 (jurajseffer)
* Testing against ES 5.4.1, 5.3.3
* Since Curator no longer supports old versions of python, convert tests to
use ``isinstance``. #973 (untergeek)
* Fix stray instance of ``is not`` comparison instead of ``!=`` #972
(untergeek)
* Increase remote client timeout to 180 seconds for remote reindex. #930
(untergeek)
* elasticsearch-py dependency bumped to 5.4.0
* Added mock dependency due to isinstance and testing requirements
* AWS ES 5.3 officially supports Curator now. Documentation has been updated
to reflect this.
* The ``_recovery`` check needs to compare using ``!=`` instead of
``is not``, which apparently does not accurately compare unicode strings.
Reported in #966 (untergeek)
* Restoring a snapshot on an exceptionally fast cluster/node can create a
race condition where a ``_recovery`` check returns an empty dictionary
``{}``, which causes Curator to fail. Added test and code to correct this.
Reported in #962. (untergeek)
* Nasty bug in schema validation fixed where boolean options or filter flags
would validate as ``True`` if non-boolean types were submitted.
Reported in #945. (untergeek)
* Check for presence of alias after reindex, in case the reindex was to an
alias. Reported in #941. (untergeek)
* Fix an edge case where an index named with `1970.01.01` could not be sorted
by index-name age. Reported in #951. (untergeek)
* Update tests to include ES 5.3.2
* Bump certifi requirement to 2017.4.17.
* Document substitute strftime symbols for doing ISO Week timestrings added
in #932. (untergeek)
* Document how to include file paths better. Fixes #944. (untergeek)
* Fixed default values for ``include_global_state`` on the restore
action to be in line with defaults in Elasticsearch 5.3
* Huge improvement to documenation, with many more examples.
* Address age filter limitations per #859 (untergeek)
* Address date matching behavior better per #858 (untergeek)
* Reindex is here! The new reindex action has a ton of flexibility. You
can even reindex from remote locations, so long as the remote cluster is
Elasticsearch 1.4 or newer.
* Added the ``period`` filter (#733). This allows you to select indices
or snapshots, based on whether they fit within a period of hours, days,
weeks, months, or years.
* Add dedicated "wait for completion" functionality. This supports health
checks, recovery (restore) checks, snapshot checks, and operations which
support the new tasks API. All actions which can use this have been
refactored to take advantage of this. The benefit of this new feature is
that client timeouts will be less likely to happen when performing long
operations, like snapshot and restore.
NOTE: There is one caveat: forceMerge does not support this, per the
Elasticsearch API. A forceMerge call will hold the client until complete,
or the client times out. There is no clean way around this that I can
discern.
* Elasticsearch date math naming is supported and documented for the
``create_index`` action. An integration test is included for validation.
* Allow allocation action to unset a key/value pair by using an empty value.
Requested in #906. (untergeek)
* Added support for the Rollover API. Requested in #898, and by countless
others.
* Added ``warn_if_no_indices`` option for ``alias`` action in response to
#883. Using this option will permit the ``alias`` add or remove to
continue with a logged warning, even if the filters result in a
``NoIndices`` condition. Use with care.
* Bumped ``click`` (python module) version dependency to 6.7
* Bumped ``urllib3`` (python module) version dependency to 1.20
* Bumped ``elasticsearch`` (python module) version dependency to 5.3
* Refactored a ton of code to be cleaner and hopefully more consistent.
* Curator now logs version incompatibilities as an error, rather than just
raising an Exception. #874 (untergeek)
* The ``get_repository()`` function now properly raises an exception instead
of returning `False` if nothing is found. #761 (untergeek)
* Check if an index is in an alias before attempting to delete it from the
alias. Issue raised in #887. (untergeek)
* Fix allocation issues when using Elasticsearch 5.1+. Issue raised in #871
(untergeek)
* Add missing repository arg to auto-gen API docs. Reported in #888
(untergeek)
* Add all new documentation and clean up for v5 specific.
* IndexList no longer checks to see if there are indices on initialization.
* Allow allocation action to unset a key/value pair by using an empty value.
Requested in #906. (untergeek)
* Added support for the Rollover API. Requested in #898, and by countless
others.
* Added ``warn_if_no_indices`` option for ``alias`` action in response to
#883. Using this option will permit the ``alias`` add or remove to
continue with a logged warning, even if the filters result in a
``NoIndices`` condition. Use with care.
* Check if an index is in an alias before attempting to delete it from the
alias. Issue raised in #887. (untergeek)
* Fix allocation issues when using Elasticsearch 5.1+. Issue raised in #871
(untergeek)
* Add missing repository arg to auto-gen API docs. Reported in #888
(untergeek)
* Update Curator to use version 5.1 of the ``elasticsearch-py`` python
module. With this change, there will be no reverse compatibility with
Elasticsearch 2.x. For 2.x versions, continue to use the 4.x branches of
Curator.
* Tests were updated to reflect the changes in API calls, which were minimal.
* Remove "official" support for Python 2.6. If you must use Curator on a
system that uses Python 2.6 (RHEL/CentOS 6 users), it is recommended that
you use the official RPM package as it is a frozen binary built on Python
3.5.x which will not conflict with your system Python.
* Use ``isinstance()`` to verify client object. #862 (cp2587)
* Prune older versions from Travis CI tests.
* Update ``certifi`` dependency to latest version
* Add version compatibility section to official documentation.
* Update docs to reflect changes. Remove cruft and references to older
versions.
* Add and increment test versions for Travis CI. #839 (untergeek)
* Make `filter_list` optional in snapshot, show_snapshot and show_indices
singleton actions. #853 (alexef)
* Fix cli integration test when different host/port are specified. Reported
in #843 (untergeek)
* Catch empty list condition during filter iteration in singleton actions.
Reported in #848 (untergeek)
* Add docs regarding how filters are ANDed together, and how to do an OR with
the regex pattern filter type. Requested in #842 (untergeek)
* Fix typo in Click version in docs. #850 (breml)
* Where applicable, replace `[source,text]` with `[source,yaml]` for better
formatting in the resulting docs.
* ``--wait_for_completion`` should be `True` by default for Snapshot
singleton action. Reported in #829 (untergeek)
* Increase `version_max` to 5.1.99. Prematurely reported in #832 (untergeek)
* Make the '.security' index visible for snapshots so long as proper
credentials are used. Reported in #826 (untergeek)
* Fixed incorrect assumption of PyPI picking up dependency for certifi. It
is still a dependency, but should not affect ``pip`` installs with an error
any more. Reported in #821 (untergeek)
* ``certifi`` is now officially a requirement.
* ``setup.py`` now forcibly includes the ``certifi`` certificate PEM file in
the "frozen" distributions (i.e., the compiled versions). The
``get_client`` method was updated to reflect this and catch it for both the
Linux and Windows binary distributions. This should `finally` put to rest
#810
-------------------------------------------------------------------
Tue Feb 14 12:55:41 UTC 2017 - artur.basiak@ts.fujitsu.com
- Python Elasticsearch curator package
