Involved Projects and Packages
Eucalyptus 2.0 is an open source Linux-based software architecture that implements scalable, efficiency-enhancing private and hybrid clouds within an organization's IT infrastructure. Eucalyptus provides Infrastructure as a Service (IaaS). This means that users can provision their own collections of resources (hardware, storage, and network) via Eucalyptus’ self-service interface on an as-needed basis. A Eucalyptus cloud is deployed across an enterprise’s “on-premise” data center and is accessed by users over enterprise intranet. Thus, with a Eucalyptus private cloud, sensitive data remains secure from external intrusion behind the enterprise firewall.
Eucalyptus was designed from the ground up to be easy to install and as non-intrusive as possible. The software framework is highly modular, with industry-standard, language-agnostic communication. Eucalyptus is also unique by providing a virtual network overlay that both isolates network traffic of different users and allows two or more clusters to appear to belong to the same Local Area Network (LAN). Eucalyptus also interoperates seamlessly with Amazon’s EC2 and S3 public cloud services and thus offers the enterprise a hybrid cloud capability.
Anyjson loads whichever is the fastest JSON module installed and provides a uniform API
regardless of which JSON implementation is used.
Originally part of carrot (http://github.com/ask/carrot/).
Celery is a distributed task queue. It was first created for Django, but is now usable from Python. It can also operate with
other languages via HTTP+JSON. It is used for executing tasks asynchronously, routed to one or more worker servers, running
concurrently using multiprocessing. It is designed to solve certain problems related to running websites demanding high-availability and performance.
It is perfect for filling caches, posting updates to twitter, mass downloading data like syndication feeds or web scraping. Use-cases are plentiful. Implementing these features asynchronously using celery is easy and fun, and the performance improvements can make it more than worthwhile.
Universal encoding detector.
Fabric is a Python (2.5 or higher) library and command-line tool for
streamlining the use of SSH for application deployment or systems
It provides a basic suite of operations for executing local or remote shell
commands (normally or via sudo) and uploading/downloading files, as well as
auxiliary functionality such as prompting the running user for input, or
HTML parser designed to follow the WHATWG HTML5 specification. The parser is designed to handle all
flavours of HTML and parses invalid documents using well-defined error handling rules compatible
with the behaviour of major desktop web browsers.
Output is to a tree structure; the current release supports output to ElementTree (including
cElementTree and lxml.etree), minidom, and a custom simpletree format.
html5lib also includes a HTML sanitizer, "treewalkers" for converting various tree formats into
streams and filters and serializers to operate on those streams.
An AMQP messaging framework for Python.
AMQP is the Advanced Message Queuing Protocol, an open standard protocol for message orientation, queuing, routing, reliability and security.
One of the most popular implementations of AMQP is RabbitMQ.
The aim of Kombu is to make messaging in Python as easy as possible by providing an idiomatic high-level interface for the AMQP protocol, and also provide proven and tested solutions to common messaging problems.
The lockfile module exports a FileLock class which provides a simple API for locking files.
Unlike the Windows msvcrt.locking function, the Unix fcntl.flock, fcntl.lockf and the
deprecated posixfile module, the API is identical across both Unix (including Linux and
Mac) and Windows platforms. The lock mechanism relies on the atomic nature of the link
(on Unix) and mkdir (on Windows) system calls.
A pure Python network address representation and manipulation library.
netaddr provides a Pythonic way of working with:
- IPv4 and IPv6 addresses and subnets (including CIDR notation);
- MAC (Media Access Control) addresses in multiple formats;
- IEEE EUI-64, OUI and IAB identifiers;
- a user friendly IP glob-style format.
Included are routines for:
- generating, sorting and summarizing IP addresses;
- converting IP addresses and ranges between various different formats;
- performing set based operations on groups of IP addresses and subnets;
- arbitrary IP address range calculations and conversions;
- querying IEEE OUI and IAB organisational information;
- querying of IP standards related data from key IANA data sources.
pip installs packages. Python packages. An easy_install replacement
Pyflakes is program to analyze Python programs and detect various errors. It works by
parsing the source file, not importing it, so it is safe to use on modules with side
effects. It's also much faster.
A Python wrapper for the Graphviz Agraph data structure.
pygraphviz can be used to create and draw networks and graphs with Graphviz.
The PyMongo distribution contains tools for interacting with MongoDB database from Python. The bson package is an implementation of the BSON format for Python. The pymongo package is a native Python driver for MongoDB. The gridfs package is a gridfs implementation on top of pymongo.
This library implements the well-behaved daemon specification of PEP 3143, "Standard daemon
A well-behaved Unix daemon process is tricky to get right, but the required steps are much the
same for every daemon program. A DaemonContext instance holds the behaviour and configured
process environment for the program; use the instance as a context manager to enter a daemon state.
Selenium Python Client Driver is a Python language binding for Selenium Remote Control.
This is a plugin for setuptools that enables git integration. Once installed, Setuptools can be told to include in a package distribution all the files tracked by git. This is an alternative to explicit inclusion specifications with MANIFEST.in.
When reading from many possibly large files in a fashion similar to random
access, it is usually the fastest and most efficient to use memory maps.
Although memory maps have many advantages, they represent a very limited
system resource as every map uses one file descriptor, whose amount is
limited per process. On 32 bit systems, the amount of memory you can have
mapped at a time is naturally limited to theoretical 4GB of memory, which
may not be enough for some applications.
The documentation can be found here: http://packages.python.org/smmap
Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.
The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll, it can handle thousands of simultaneous standing connections, which means it is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features — every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)
virtualenv is a tool to create isolated Python environments.
virtualenvwrapper is a set of extensions to Ian Bicking's virtualenv tool. The extensions include wrappers for creating and deleting virtual environments and otherwise managing your development workflow, making it easier to work on more than one project at a time without introducing conflicts in their dependencies.
* Organizes all of your virtual environments in one place.
* Wrappers for creating, copying and deleting environments, including user-configurable hooks.
* Use a single command to switch between environments.
* Tab completion for commands that take a virtual environment as argument.
* User-configurable hooks for all operations.
* Plugin system for more creating sharable extensions.