rse.main.parsers package

Submodules

rse.main.parsers.base module

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

class rse.main.parsers.base.Capturing[source]

Bases: object

capture output from stdout and stderr into capture object. This is based off of github.com/vsoch/gridtest but modified to write files. The stderr and stdout are set to temporary files at the init of the capture, and then they are closed when we exit. This means expected usage looks like:

with Capturing() as capture:: process = subprocess.Popen(…)

And then the output and error are retrieved from reading the files: and exposed as properties to the client:

capture.out capture.err

And cleanup means deleting these files, if they exist.

cleanup()[source]

property err: Return error stream. Returns empty string if empty or doesn’t exist. Returns (str) : error stream written to file

property out: Return output stream. Returns empty string if empty or doesn’t exist. Returns (str) : output stream written to file

set_stderr()[source]

set_stdout()[source]

class rse.main.parsers.base.ParserBase(uid=None)[source]

Bases: object

A parser base exists to extract and format repository metadata.

capture(cmd)[source]

capture is a helper function to capture a shell command. We use Capturing and then save attributes like the pid, output, error to it, and return to the calling function. For example:

capture = self.capture_command(cmd) self.pid = capture.pid self.returncode = capture.returncode self.out = capture.output self.err = capture.error

export()[source]: return data as json. This is intended to save to the software database. Any important parser specific metadata should be added to self.data

get_description(data)[source]: a common function for a parser to return a description.

get_metadata(uri, **kwargs)[source]: The get_metadata function should take a general URI for a parser and populate the self.data

get_setting(key, default=None)[source]: Get a setting, meaning that we first check the environment, then the config file, and then (if provided) a default.

get_url(data)[source]: a common function for a parser to return the html url for the upper level of metadata

load(data)[source]: If a repository has already been instantiated, we might want to load data into a parser to interact with it

name = 'base'

set_uid(uid)[source]: Given a unique resource identifier, set it for the parser

summary()[source]

rse.main.parsers.github module

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

class rse.main.parsers.github.GitHubParser(uid=None, **kwargs)[source]

Bases: rse.main.parsers.base.ParserBase

get_avatar(data=None)[source]: a common function for a parser to return an image.

get_description(data=None)[source]: a common function for a parser to return a description.

get_metadata(uri=None)[source]

Retrieve repository metadata. The common metadata (timestamp) is added by the software repository parser, and here we need to ensure that the url field is populated with a correct url.

Arguments: uri (str) : a repository uri string to override one currently set

get_url(data=None)[source]: a common function for a parser to return the html url for the upper level of metadata

load_secrets()[source]: load secrets, namely the GitHub token

matchstring = 'github'

name = 'github'

Module contents

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

rse.main.parsers.get_named_parser(name, uri=None, config=None)[source]: get a named parser, meaning determining based on name and not uri

rse.main.parsers.get_parser(uri, config=None)[source]: get parser will return the correct parser depending on a uri

rse.main.parsers.matches(Parser, uri)[source]: Given a unique resource identifier, determine if it matches a regular expression used for a parser (or not)