Internal API¶

pipsalabim¶

pipsalabim is just black magic.

Pip Sala Bim is a package that studies the codebase of your project in search for internal and external imports. It then discards the imports that are satisfied with internal code or with the standard library and finally searches the PyPIContents index to list which packages satisfy your imports.

pipsalabim.cli module¶

pipsalabim.cli is a module for handling the command line interface.

This module handles the commands for using Pip Sala Bim. It also parses parameters, show help, version and controls the logging level.

pipsalabim.cli.commandline(argv=None)[source]¶

Configure ArgumentParser to accept custom arguments and commands.

Parameters:	argv – a list of commandline arguments like `sys.argv`. For example: ['-v', '--loglevel', 'INFO']
Returns:	a `Namespace` object.

New in version 0.1.0.

pipsalabim.cli.main(argv=None)[source]¶

Handle arguments and commands.

Parameters:	argv – a list of commandline arguments like `sys.argv`. For example: ['-v', '--loglevel', 'INFO']
Returns:	an exit status.

New in version 0.1.0.

pipsalabim.api package¶

pipsalabim.api is a package containing all the cli commands.

This package contains each command of Pip Sala Bim organized in modules.

pipsalabim.api.report module¶

pipsalabim.api.report is a module implementing the report command.

This module contains the logic to examine your source code, extract internal and external imports, and finally determine which external PyPI packages you need to install in order to satisfy dependencies.

pipsalabim.api.report.ask_multiple_pypi(datadict)[source]¶

Ask the user about which PyPI package will use to satisfy an import.

Parameters:	datadict – a dictionary containing modules as keys and a list as values.
Returns:	an updated `datadict` with the answered information from user.

New in version 0.1.0.

pipsalabim.api.report.get_imports(pkgdata)[source]¶

List modules imported inside of packages provided in pkgdata.

Parameters:	pkgdata – a list of tuples containing the name of a package and the directory where its located.
Returns:	a list of the modules imported according to the list of packages provided in `pkgdata`.

New in version 0.1.0.

pipsalabim.api.report.get_messages(datadict)[source]¶

Generate messages for each type of module in datadict.

Parameters:	datadict – a dictionary containing modules as keys and a list as values.
Returns:	a dictionary containing messages for each type of module.

New in version 0.1.0.

pipsalabim.api.report.get_module_datadict(basedir)[source]¶

Process the current directory to get data from packages and modules.

Parameters: basedir – a string containing a path to the directory to be analized.

Returns:

a dictionary containing information for each imported module. Like:

{
    'module_a': ['LOCAL'],
    'module_b': ['pypi_package_1'],
    'module_c': ['pypi_package_1', 'pypi_package_2'],
    'module_d': [],
    'module_e': ['STDLIB2.6', 'STDLIB2.7', 'STDLIB3.5'],
    'module_f': ['STDLIB2.7'],
}

New in version 0.1.0.

pipsalabim.api.report.get_modules(pkgdata)[source]¶

List modules inside packages provided in pkgdata.

Parameters:	pkgdata – a list of tuples containing the name of a package and the directory where its located.
Returns:	a list of the modules according to the list of packages provided in `pkgdata`.

New in version 0.1.0.

pipsalabim.api.report.get_package_dirs(path)[source]¶

List directories containing python packages on path.

Parameters:	path – a path pointing to a directory containing python code.
Returns:	a list containing directories of packages.

New in version 0.1.0.

pipsalabim.api.report.get_packages(path)[source]¶

List packages living in path with its directory.

Parameters:	path – a path pointing to a directory containing python code.
Returns:	a list of tuples containing the name of the package and the package directory. For example: [ ('package_a', '/path/to/package_a'), ('package_b.module_b', '/path/to/package_b/module_b'), ('package_c.module_c', '/path/to/package_c/module_c') ]

New in version 0.1.0.

pipsalabim.api.report.main(**kwargs)[source]¶

Generate a report to inform about PyPI dependencies.

This command will search your code for unsatisfied dependencies by looking at your import statements. If an import is not satisfied by internal modules or the standard library, then it will query the PyPI module index provided by PyPIContents.

Sometimes, more than one PyPI package will provide the missing module and in such cases, you will be asked to select one from a list of options.

If Pip Sala Bim fails to find a package providing the module you need, it will report it back to you.

Returns:	an exit status.

New in version 0.1.0.

pipsalabim.api.update module¶

pipsalabim.api.update is a module implementing the update command.

Pip Sala Bim needs to keep a local copy of the PyPIContents database to function properly. This command takes care of maintaining it up-to-date.

Please note that this command must be executed manually.

pipsalabim.api.update.download_json(datafile, dataurl, istarred)[source]¶

Download a json file from dataurl, store to datafile and report.

Parameters:	datafile – a string containing a path to store the contents of the file downloaded from `dataurl`. dataurl – a string containing a url to a file.
Returns:	`True` if operations went OK, `False` if not.

New in version 0.1.0.

pipsalabim.api.update.main(**kwargs)[source]¶

Update databases from PyPIContents.

https://raw.githubusercontent.com/LuisAlejandro/pypicontents-build/master/stdlib.json .. _pypi.json.xz: https://raw.githubusercontent.com/LuisAlejandro/pypicontents-build/master/pypi.json.xz .. _PyPIContents project: https://github.com/LuisAlejandro/pypicontents-build

This function downloads the standard library modules index (stdlib.json) and the PyPI modules index (`pypi.json.xz`_) from the `PyPIContents project`_ to keep a local copy of these files, which are critical to PiP Sala Bim.

This command makes no comparing or checking prior to download.

Returns:	an exit status.

New in version 0.1.0.

pipsalabim.core package¶

pipsalabim.core contains the core functions of the application.

This package contains functions for managing the download of files, exploring and obtaining imported modules, logging and other minor file operations.

pipsalabim.core.imports module¶

pipsalabim.core.imports studies the code searching for imports.

This module has advanced programming to inspect and locate imported modules in python source code. It uses python’s own Abstract Syntax Trees (AST) to do such operations.

class pipsalabim.core.imports.ImportVisitor[source]¶

AST visitor for grabbing the import statements.

ImportVisitor is a node visitor class that walks the abstract syntax tree and calls a visitor function for every import statement found.

Per default the visitor functions for the nodes are 'visit_' + class name of the import node (lowercase). This visitor only has methods for processing Import and ImportFrom types of nodes. When any other type of node is entered, the generic_visit visitor is used instead.

This visitor produces a list of tuples like:

[
    ('MODULE', 'LEVEL'),
    ('MODULE', 'LEVEL'),
    ('MODULE', 'LEVEL'),
    ('MODULE', 'LEVEL'),
    ('MODULE', 'LEVEL')
]

MODULE: a string containing the name of the imported module

(does not include the package to which it belongs).
LEVEL is the relative level of the imported module.
- 0 is the level of a simple import like:
  import MODULE from MODULE import NAME
- 1 is the level of a relative import in the same package, for example:
  import .MODULE from .MODULE import NAME from . import NAME
- 2 is the level of a relative import in the parent package, for example:
  import ..MODULE from ..MODULE import NAME from .. import NAME

__init__()[source]¶

Initialize this ImportVisitor.

Sets initial empty values for self.modules which will help storing modules.

Returns:	an `ImportVisitor` instance.

New in version 0.1.0.

modules = None¶: Attribute modules (list): Stores modules as they are found by the visitor methods.

visit(node)[source]¶

Visit a node.

Parameters:	node – an `ast` object representing a python statement.
Returns:	a reference to `visit_import` if the node type is `Import`, to `visit_importfrom` if the node type is `ImportFrom` or `generic_visit` for any other node.

New in version 0.1.0.

visit_import(node)[source]¶

Append node names to self.modules.

Parameters:	node – an `ast` object representing a python statement.

New in version 0.1.0.

visit_importfrom(node)[source]¶

Append node names and levels to self.modules.

Parameters:	node – an `ast` object representing a python statement.

New in version 0.1.0.

pipsalabim.core.imports.find_imports(package, filename)[source]¶

Get a list of modules extracted from import statements.

Parameters:	package – a string containing a python package to which `filename` belongs. filename – a string containing a path to a python source code file.
Returns:	a list of modules in absolute form.

New in version 0.1.0.

pipsalabim.core.imports.parse_python_source(filename)[source]¶

Parse the file filename and convert it to AST.

Parameters:	filename – a string containing a path to a python source code file.
Returns:	an AST object or `None` if the file has a syntax error.

New in version 0.1.0.

pipsalabim.core.logger module¶

pipsalabim.core.logger is the global application logging module.

All modules use the same global logging object. No messages will be emitted until the logger is started.

class pipsalabim.core.logger.ControlableLogger(name=None)[source]¶

This class represents a logger object that can be started and stopped.

It has a start method which allows you to specify a logging level. The stop method halts output.

__init__(name=None)[source]¶

Initialize this ControlableLogger.

The name defaults to the application name. Loggers with the same name refer to the same underlying object. Names are hierarchical, e.g. ‘parent.child’ defines a logger that is a descendant of ‘parent’.

Parameters:	name – a string containig the logger name.
Returns:	a `ControlableLogger` instance.

New in version 0.1.0.

disabled = None¶: Attribute disabled (boolean): Stores the current status of the logger.

formatstring = None¶: Attribute formatstring (string): Stores the string that will be used to format the logger output.

loglevel(level='WARNING')[source]¶

Set the log level for this logger.

Messages less than the given priority level will be ignored. The default level is ‘WARNING’, which conforms to the *nix convention that a successful run should produce no diagnostic output. Available levels and their suggested meanings:

NOTSET: all messages are processed.
DEBUG: output useful for developers.
INFO: trace normal program flow, especially external

interactions.
WARNING: an abnormal condition was detected that might need

attention.
ERROR: an error was detected but execution continued.
CRITICAL: an error was detected and execution was halted.

Parameters:	level – a string containing the desired logging level.

New in version 0.1.0.

start()[source]¶

Start logging with this logger.

Until the logger is started, no messages will be emitted. This applies to all loggers with the same name and any child loggers.

New in version 0.1.0.

stop()[source]¶

Stop logging with this logger.

Remove available handlers and set active attribute to False.

New in version 0.1.0.

pipsalabim.core.utils module¶

pipsalabim.core.utils is a utility module.

This module contains several utilities to process information coming from the other modules.

pipsalabim.core.utils.chunk_read(response, chunk_size=8192, report_hook=None)[source]¶

Download a file by chunks.

Parameters:	response – a file object as returned by `urlopen`. chunk_size – an integer representing the size of the chunks to be downloaded at a time. report_hook – a function to report the progress of the download.
Returns:	a blob containing the downloaded file.

New in version 0.1.0.

pipsalabim.core.utils.chunk_report(downloaded, total)[source]¶

Print the progress of a download.

Parameters:	downloaded – an integer representing the size (in bytes) of data downloaded so far. total – an integer representing the total size (in bytes) of data that needs to be downloaded.

New in version 0.1.0.

pipsalabim.core.utils.custom_sys_path(new_sys_path)[source]¶

Context manager to momentarily change sys.path.

Parameters:	new_sys_path – a list of paths to overwrite `sys.path`.

New in version 0.1.0.

pipsalabim.core.utils.fill_with_local(datadict, modules)[source]¶

Fill datadict if module is found in modules.

Parameters:	datadict – a dictionary containing modules as keys and a list as values. modules – a list of modules present in the local python source code.
Returns:	a dictionary containing information about the location of each imported module.

New in version 0.1.0.

pipsalabim.core.utils.fill_with_pypi(datadict, pypidata)[source]¶

Fill datadict with modules from pypidata if found.

Parameters:	datadict – a dictionary containing modules as keys and a list as values. pypidata – a dictionary with the PyPIContents database.
Returns:	an updated dictionary containing information about the location of each imported module.

New in version 0.1.0.

pipsalabim.core.utils.fill_with_stdlib(datadict, stdlibdata)[source]¶

Fill datadict with modules from stdlibdata if found.

Parameters:	datadict – a dictionary containing modules as keys and a list as values. stdlibdata – a dictionary containing the modules of the standard library of each python version.
Returns:	a dictionary containing information about the location of each imported module.

New in version 0.1.0.

pipsalabim.core.utils.find_files(path=None, pattern='*')[source]¶

Locate all the files matching the supplied pattern in path.

Locate all the files matching the supplied filename pattern in and below the supplied root directory. If no pattern is supplied, all files will be returned.

Parameters:	path – a string containing a path where the files will be looked for. pattern – a string containing a regular expression.
Returns:	a list of files matching the pattern within path (recursive).

New in version 0.1.

pipsalabim.core.utils.is_valid_path(path)[source]¶

Test if path is a valid python path.

Parameters:	path – a string containing a path.
Returns:	`True` if `path` is a valid python path. `False` otherwise.

New in version 0.1.0.

pipsalabim.core.utils.list_files(path=None, pattern='*')[source]¶

List files on path (non-recursively).

Locate all the files matching the supplied filename pattern in the first level of the supplied path. If no pattern is supplied, all files will be returned.

Parameters:	path – a string containing a path where the files will be looked for. pattern – a string containing a regular expression.
Returns:	a list of files matching the pattern within the first level of path (non-recursive).

New in version 0.1.0.

pipsalabim.core.utils.remove_sys_modules(remove)[source]¶

Context manager to momentarily remove modules from sys.modules.

Parameters:	remove – a list of modules to remove from `sys.modules`.

New in version 0.1.0.