pepper package

Submodules

pepper.preprocessor

This module contains the functions necessary to run only the preprocessor

It primarily serves as the entry point to Pepper

pepper.preprocessor.get_args()

pepper.preprocessor.main(args=None)

pepper.symbol_table

The Symbol Table module implements a class to track declarations and usages of identifiers

pepper.symbol_table.FILE_STACK = []

The stack of files we’re reading from

pepper.symbol_table.IFDEF_STACK = []

The stack of ifdef/ifndef/if control structures we’re processing

pepper.symbol_table.LINUX_DEFAULTS = ['/usr/include/c++/7', '/usr/include/x86_64-linux-gnu/c++/7', '/usr/include/c++/7/backward', '/usr/lib/gcc/x86_64-linux-gnu/7/include', '/usr/local/include', '/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed', '/usr/include/x86_64-linux-gnu', '/usr/include']

The default linux paths to search for includes-

class pepper.symbol_table.MacroExpansion(name, expansion, args=None)

Bases: object

Expands an identifier into a macro expansion, possibly with arguments

expand(args=None)

Expand macro, maybe with args

preprocess(lines)

exception pepper.symbol_table.PepperInternalError(msg=None)

Bases: Exception

exception pepper.symbol_table.PepperSyntaxError(msg=None)

Bases: Exception

pepper.symbol_table.SYSTEM_INCLUDE_PATHS = ['/usr/include/c++/7', '/usr/include/x86_64-linux-gnu/c++/7', '/usr/include/c++/7/backward', '/usr/lib/gcc/x86_64-linux-gnu/7/include', '/usr/local/include', '/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed', '/usr/include/x86_64-linux-gnu', '/usr/include']

The list of paths to search when doing a system include

pepper.symbol_table.TABLE = {}

The global symboltable

pepper.symbol_table.TRIGGER_INTERNAL_ERROR = False

Switch to test internal error handling

pepper.abstract_symbol_tree module

This is the abstract symbol tree for PEPPr.

The parser will build the actual tree, so this is really more of a library of nodes that may be used within the tree.

class pepper.abstract_symbol_tree.ASCIILiteralNode(children)

Bases: pepper.abstract_symbol_tree.PrimitiveNode

class pepper.abstract_symbol_tree.IdentifierNode(children, args=None, variadic=False)

Bases: pepper.abstract_symbol_tree.Node

preprocess(lines=None)

class pepper.abstract_symbol_tree.LinesNode(children=None)

Bases: pepper.abstract_symbol_tree.Node

preprocess(lines=None)

class pepper.abstract_symbol_tree.NewlineNode(children)

Bases: pepper.abstract_symbol_tree.PrimitiveNode

preprocess(lines)

class pepper.abstract_symbol_tree.Node(name='Node', children=None)

Bases: object

preprocess(lines)

class pepper.abstract_symbol_tree.PreprocessingNumberNode(children)

Bases: pepper.abstract_symbol_tree.PrimitiveNode

class pepper.abstract_symbol_tree.PreprocessorDirectiveNode(children)

Bases: pepper.abstract_symbol_tree.Node

class pepper.abstract_symbol_tree.PreprocessorIncludeNode(children, system_include=False)

Bases: pepper.abstract_symbol_tree.Node

preprocess(lines)

This will be a lie for a while. I’ll have to fix it later.

search_system_includes()

class pepper.abstract_symbol_tree.PrimitiveNode(name, children)

Bases: pepper.abstract_symbol_tree.Node

preprocess(lines=None)

class pepper.abstract_symbol_tree.StringLiteralNode(children)

Bases: pepper.abstract_symbol_tree.PrimitiveNode

class pepper.abstract_symbol_tree.WhiteSpaceNode(children)

Bases: pepper.abstract_symbol_tree.PrimitiveNode

pepper.lexer module

This is the lexer for PEPPr

It’s responsible for tokenizing the incoming character stream. The Parser will ingest the token stream and build a tree, which will in turn produce actual c++ or c code.

pepper.lexer.get_args()

pepper.lexer.lex(lines, debug_mode=False)

Takes a single string, containing newlines, that’s the entire input

pepper.lexer.main()

pepper.lexer.t_COMMENT(t)

s//.*

pepper.lexer.t_COMMENT_NO_WHITESPACE(t)

//.*

pepper.lexer.t_IDENTIFIER(t)

([_a-zA-Z][_a-zA-Z0-9]*(…)?)|(…)

pepper.lexer.t_LONG_COMMENT_START(t)

/*

pepper.lexer.t_NEWLINE(t)

n

pepper.lexer.t_PREPROCESSING_KEYWORD_DEFINE(t)

#defineb

pepper.lexer.t_PREPROCESSING_KEYWORD_ELSE(t)

#elseb

pepper.lexer.t_PREPROCESSING_KEYWORD_ENDIF(t)

#endifb

pepper.lexer.t_PREPROCESSING_KEYWORD_IF(t)

#ifb

pepper.lexer.t_PREPROCESSING_KEYWORD_IFDEF(t)

#ifdefb

pepper.lexer.t_PREPROCESSING_KEYWORD_IFNDEF(t)

#ifndefb

pepper.lexer.t_PREPROCESSING_KEYWORD_INCLUDE(t)

#includeb

pepper.lexer.t_PREPROCESSING_KEYWORD_PY(t)

#pyb

pepper.lexer.t_PREPROCESSING_NUMBER(t)

.?[0-9]([0-9]|(e+)|(e-)|(E+)|(E-)|(p+)|(p-)|(P+)|(P-)|[a-zA-Z])*

pepper.lexer.t_STRING_LITERAL(t)

(‘((\[‘tn])|[^’\])*’)|(“((\[“tn])|[^”\])*”)

pepper.lexer.t_SYSTEM_INCLUDE_LITERAL(t)

<[^’”<>]*?>

pepper.lexer.t_WHITESPACE(t)

[t ]

pepper.lexer.t_comment_BLOCK_COMMENT_END(t)

*/

pepper.lexer.t_comment_NEWLINE(t)

n

pepper.lexer.t_comment_error(t)

pepper.lexer.t_comment_ignore_anything_else(t)

.+?

pepper.lexer.t_error(t)

pepper.parser module

This is the Parser module for Pepper

This module impelements the grammar for the preprocessor language, comprised of tokens from the Lexer module. This module implements a main function, but this is only for debugging and will be removed on release.

pepper.parser.get_args()

pepper.parser.main()

pepper.parser.p_code_expression_to_safe(p)

code_expression : safe_code_expression

pepper.parser.p_define_expansion(p)

macro_expansion : code_expressions

pepper.parser.p_define_expression_no_args(p)

define_expression : PREPROCESSING_KEYWORD_DEFINE WHITESPACE IDENTIFIER WHITESPACE macro_expansion

pepper.parser.p_define_expression_no_expansion(p)

define_expression : PREPROCESSING_KEYWORD_DEFINE WHITESPACE IDENTIFIER

pepper.parser.p_define_expression_some_args(p)

define_expression : PREPROCESSING_KEYWORD_DEFINE WHITESPACE IDENTIFIER ‘(‘ identifier_list ‘)’ maybe_space macro_expansion

pepper.parser.p_else_expression(p)

else_expression : PREPROCESSING_KEYWORD_ELSE

pepper.parser.p_endif_expression(p)

endif_expression : PREPROCESSING_KEYWORD_ENDIF

pepper.parser.p_error(p)

pepper.parser.p_expression_list_empty(p)

list_of_expressions :

pepper.parser.p_expression_list_multiple(p)

list_of_expressions : list_of_expressions ‘,’ safe_code_expressions

pepper.parser.p_expression_list_singleton(p)

list_of_expressions : safe_code_expressions

pepper.parser.p_expression_to_list_of_something(p)

code_expression_parenthetical : ‘(‘ list_of_expressions ‘)’

pepper.parser.p_expression_to_string_lit(p)

safe_code_expression : STRING_LITERAL

pepper.parser.p_expressions(p)

code_expressions : code_expressions code_expression

pepper.parser.p_expressions_empty(p)

code_expressions :

pepper.parser.p_expressions_to_single(p)

code_expressions : code_expression

pepper.parser.p_identifier_call(p)

safe_code_expression : IDENTIFIER code_expression_parenthetical

pepper.parser.p_identifier_list_empty(p)

identifier_list :

pepper.parser.p_identifier_list_multiple(p)

identifier_list : identifier_list ‘,’ maybe_space IDENTIFIER

pepper.parser.p_identifier_list_singleton(p)

identifier_list : IDENTIFIER

pepper.parser.p_ifdef_expression(p)

ifdef_expression : PREPROCESSING_KEYWORD_IFDEF WHITESPACE IDENTIFIER

pepper.parser.p_ifndef_expression(p)

ifndef_expression : PREPROCESSING_KEYWORD_IFNDEF WHITESPACE IDENTIFIER

pepper.parser.p_include_expression(p)

preprocessor_expression : include_expression

define_expression

ifdef_expression

ifndef_expression

endif_expression

else_expression

pepper.parser.p_include_expression_disambiguation(p)

include_expression : include_expression_file

include_expression_system

pepper.parser.p_include_expression_file(p)

include_expression_file : PREPROCESSING_KEYWORD_INCLUDE WHITESPACE STRING_LITERAL

pepper.parser.p_include_expression_system(p)

include_expression_system : PREPROCESSING_KEYWORD_INCLUDE WHITESPACE SYSTEM_INCLUDE_LITERAL

pepper.parser.p_line_to_statement(p)

line : statement NEWLINE

pepper.parser.p_lines_nonempty(p)

lines : lines line

pepper.parser.p_maybe_whitespace_none(p)

maybe_space :

pepper.parser.p_maybe_whitespace_some(p)

maybe_space : WHITESPACE

pepper.parser.p_pepper_directive(p)

pepper_directive : preprocessor_expression

pepper.parser.p_program(p)

program : lines statement

pepper.parser.p_safe_code_expression_to_parens(p)

safe_code_expression : code_expression_parenthetical

pepper.parser.p_safe_code_expressions_ascii_literal(p)

safe_code_expression : ‘<’

‘>’

‘+’

‘-‘

‘%’

‘^’

‘&’

‘*’

‘{‘

‘}’

‘[‘

‘]’

‘=’

‘;’

‘:’

‘#’

‘.’

‘?’

‘~’

pepper.parser.p_safe_expressions(p)

safe_code_expressions : safe_code_expressions safe_code_expression

pepper.parser.p_safe_expressions_empty(p)

safe_code_expressions :

pepper.parser.p_statement_to_ascii_literal(p)

code_expression :

‘,’

‘(‘

‘)’

pepper.parser.p_statement_to_code_expression(p)

statement : code_expressions

pepper.parser.p_statement_to_identifier(p)

safe_code_expression : IDENTIFIER

pepper.parser.p_statement_to_pepper_directive(p)

statement : pepper_directive

pepper.parser.p_statement_to_preprocessing_number(p)

safe_code_expression : PREPROCESSING_NUMBER

pepper.parser.p_statements_empty(p)

lines :

pepper.parser.p_whitespace_unsafe(p)

safe_code_expression : WHITESPACE

pepper.parser.parse(source, debug_mode=False)

Module contents