Writting new commands is quite simple and straightforward process. The only one harder or trickier part is grammar definition for the command, which requires careful design and thinking.
Because PowerConsole uses PyParsing to process commands to Python function calls, you should start by familiarize yourself with it. PyParsing provides a library of classes that you can use to construct the grammar directly in Python code. The grammar representation is quite readable, owing to the self-explanatory class names, and the use of ‘+’, ‘|’ and ‘^’ operator definitions.
The PyParsing website and especially the distribution package itself contains documentation and quite a lot of examples.
You will need setuptools installed. If you don’t have setuptools yet, you can install it by downloading and running ez_setup.py.
To create a new command you need to do next steps:
Create a new project (directory) for your command package.
Create a new Python module for your commands. While you’re free how to name the module or structure your sources in modules and module packages, you should consider to put your modules into namespace package ‘pwc’. To do so, simply create a subdirectory named pwc in your project and place the file __init__.py with next content to it:
__import__('pkg_resources').declare_namespace(__name__)
Next create your module(s) in this directory. The module name needs a little bit of thinking to avoid name collisions with command modules created by other developers.
Grammar is defined in __init__() method of your command class using PyParsing building blocks. These blocks compose a parsing tree, where some nodes may play a special role. You can define your grammar as you see fit for your purpose, but to define a grammar that could be sucessfuly used by PowerConsole, you have to follow few rules.
One example is better than thousand words, so here are some example grammars:
This example shows the definition of built-in command LIST.
Grammar:
LIST [<attribute>[,<attribute>] IN] <expression>
Definition:
from pyparsing import *
from pwc.base import *
class cmdList(Command):
def __init__(self,controller):
super(cmdList,self).__init__(controller)
# Grammar
self.keyList = self._makeNameNode(CaselessKeyword('list'))
keyAttr = delimitedList(self.IDENT,combine=True).setResultsName('attr') + CaselessKeyword('in')
# LIST [<attribute>,<attribute> IN] <expression>
self.cmdList = self.keyList + Optional(keyAttr) + self.EXPR.setResultsName('expr')
self.cmdList.setParseAction(self._compile)
def _getGrammar(self):
return self.cmdList
def _getCmdKeyword(self):
return self.keyList
def execute(self,expr,attr=None):
# Implemenetation
pass
The check grammar node is caseless keyword ‘list’. It’s also defined as name node. Because the expression is last element of the grammar and could potentially span over multiple lines, we can use inherited definition EXPR that’s defined in pwc.base.Command for your convenience as follows:
CHUNK = CharsNotIn('\n')
EXPR = White().setName('<expr>').suppress() + OneOrMore(CHUNK + (
(Optional(lineEnd.setParseAction(replaceWith(' '))) +
FollowedBy(CHUNK)) ^
Optional(lineEnd.suppress())
)
).setName('expr')
We assigned the results name ‘expr’ to it, because our execute method expects parameter with this name. Note that line end is replaced with space, so the expression is compiled as it would be on single line. The full grammar is cmdList and has assigned parse action pwc.base.Command._compile().
Here is an example of simple command with many parameters, some of them optional.
Grammar:
CONNECT 'dsn or db' [HOST host][USER 'user'][PASSWORD 'password'] [CACHE int ][ROLE 'role'][AS var]
Definition:
def __init__(self, controller):
...
self.keyConnect = self._makeNameNode(CaselessKeyword('connect'))
# we don't need to keep next tokens as instance attributes
keyUser = CaselessKeyword('user')
keyPassword = CaselessKeyword('password')
keyCache = CaselessKeyword('cache')
keyRole = CaselessKeyword('role')
keyAs = CaselessKeyword('as')
keyHost = CaselessKeyword('host')
optUser = keyUser + QuotedString("'").setResultsName('user').setFailAction(self._fail)
optPassword = keyPassword + QuotedString("'").setResultsName('password').setFailAction(self._fail)
optCache = keyCache + self.INTEGER.setResultsName('cache').setFailAction(self._fail)
optRole = keyRole + QuotedString("'").setResultsName('role').setFailAction(self._fail)
optAs = keyAs + self.IDENT.setResultsName('intoVar').setFailAction(self._fail)
optHost = keyHost + self.IDENT.setResultsName('host').setFailAction(self._fail)
# CONNECT 'dsn' [USER 'user'][PASSWORD 'password']
# [CACHE int ][ROLE 'role'][AS var]
self.cmdConnect = self.keyConnect + self.FILENAME.setResultsName('db') \
+ Optional(optHost) + Optional(optUser) + Optional(optPassword) \
+ Optional(optCache) + Optional(optRole) + Optional(optAs)
self.cmdConnect.setParseAction(self._compile)
def _getGrammar(self):
return self.cmdConnect
def _getCmdKeyword(self):
return self.keyConnect
def execute(self,db,**kwargs):
...
The check grammar node is caseless keyword ‘connect’. It’s also defined as name node. optXXX are nodes for command options. The value part has assigned fail action to inherited _fail() to get nice error reporting if value is not entered. We also must assign a ResultsName for each value to name that our execute methoud would recognize. For file name we could utilize the inherited FILENAME node that’s defined as follows:
FILENAME = (Word(alphas +
alphas8bit + nums + '_/\-:.').setName('filename') |
QuotedString("'")).setName('filename')
The full grammar is cmdConnect and has assigned parse action pwc.base.Command._compile().
Sometimes, you want to handle several commands by single execute method (i.e. command class). For example it’s pointless to implement each SQL command as separate command class. In case of SQL, we don’t even need to define the full grammar, because SQL commands are terminated, we just need to define the check grammar.
Grammar:
Selected SQL commands
Definition:
usesTerminator = True
def __init__(self,controller):
...
# Grammar
# SQL commands
keyCommit = CaselessKeyword('commit')
keyRollback = CaselessKeyword('rollback')
keyAlter = CaselessKeyword('alter')
keyCreate = CaselessKeyword('create')
keyDelete = CaselessKeyword('delete')
keyDrop = CaselessKeyword('drop')
keyExecute = CaselessKeyword('execute')
keyGrant = CaselessKeyword('grant')
keyInsert = CaselessKeyword('insert')
keyRecreate = CaselessKeyword('recreate')
keyRevoke = CaselessKeyword('revoke')
keySavepoint= CaselessKeyword('savepoint')
keySelect = CaselessKeyword('select')
keyUpdate = CaselessKeyword('update')
keySet = CaselessKeyword('set')
# Second level keywords
keyGenerator = CaselessKeyword('generator')
keyStatistics = CaselessKeyword('statistics')
# Composite SQL commands
cmdSetGen = (keySet + keyGenerator)
cmdSetStat = (keySet + keyStatistics)
# Grammar trick to sink all SQL commands into this class
self.keySQL = self._makeNameNode(Empty())
self.cmdAll = (keySelect ^ keyCommit ^ keyRollback ^ keyAlter ^ keyCreate ^
keyDelete ^ keyDrop ^ keyExecute ^ keyGrant ^ keyInsert ^
keyRecreate ^ keyRevoke ^ keySavepoint ^ keyUpdate ^
cmdSetGen ^ cmdSetStat
)
# Multiline SQL command
SQL = OneOrMore(CHUNK + (
(Optional(lineEnd.setParseAction(replaceWith('\\n'))) +
FollowedBy(CHUNK)) ^
Optional(lineEnd.suppress())
)
)
self.cmdSQL = self.keySQL + Combine(self.cmdAll + SQL).setResultsName('sql')
self.cmdSQL.setParseAction(self._compile)
def _getGrammar(self):
return self.cmdSQL
def _getCmdKeyword(self):
return self.keySQL + self.cmdAll
def execute(self,sql):
...
First, we need to set the class attribute usesTerminator to True, because SQL commands use command terminator. Next we define the check grammar cmdAll that would recognize all SQL commands we want to handle. The full grammar would be empty name node followed by SQL command that would consist from our SQL keywords + rest of the command that may span multiple lines (up to the terminator sequence) combined into one text node with results name ‘sql’. Note that line ends are preserved in multiline command. That’s ok, because the whole command would be quoted as string parameter, and we want to preserve the line ends in SQL command (for example when we’ll define stored procedure, it does matter a lot).
Note
We’re using empty name node keySQL in both grammars, because bot must contain the name node.
Sometimes you want to define command with keyword(s) that may very likely collide with other commands created by other command developers. In that case, you need to make it unique by introducing domain keyword(s). For example Firebird QA command pack contains command RUN, that will collide with standard command RUN. So all QA commands use keyword QA followed by command keyword.
Grammar:
QA RUN [FORMAT {FULL | BRIEF | BATCH | STATS | NONE}] [ARCHIVE | test-name]
Definition:
def __init__(self,controller):
...
# Grammar
self.keyCmdName = self._makeNameNode(Empty())
testName = Word(alphas,alphanums+'_-.')
# QA RUN [FORMAT {FULL | BRIEF | BATCH | STATS | NONE}] [ARCHIVE | test-name]
self.keyQARun = self._makeNameNode(Combine(
CaselessKeyword('QA') + White() +
CaselessKeyword('RUN')))
self.cmdQARun = self.keyCmdName + self.keyQARun + \
Optional(CaselessKeyword('FORMAT') +
oneOf('FULL BRIEF BATCH STATS NONE',caseless=True).setResultsName('format')) + \
Optional(CaselessKeyword('ARCHIVE').setResultsName('archive') | testName.setResultsName('test')) + \
Optional(lineEnd.suppress())
self.cmdQARun.setParseAction(self._compile)
def _getGrammar(self):
return self.cmdQARun
def _getCmdKeyword(self):
return self.keyQARun
def execute(self,format='brief',archive=None,test=None):
...
In this case you’ll have to work around the “limitation” of PyParsing and how translation of command to python call works. The _compile method requires that there is a token ‘cmd’ that contains command name so it could construct the correct _XXX_execute call. This node is also used by execution engine to identify the command and look it up in command table. Hover, if your grammar node that defines the name is “structural” node like Combine, it doesn’t show up in parameters list and the routine fails. To work around that, you need to define two name nodes. The obvious one for your keywords in check grammar:
self.keyQARun = self._makeNameNode(Combine(CaselessKeyword('QA') + White() + CaselessKeyword('RUN')))
and dummy one that will appear in full grammar:
self.keyCmdName = self._makeNameNode(Empty())
self.cmdQARun = self.keyCmdName + self.keyQARun + \
...
Grammar definition could be tricky and it may happen you would not get it right from the start, especially when you would use any definition that can potentially match more than expected (some grammar nodes are “greedy” by design, like EXPR or ARG). In that case you would need to debug your grammar. To do so, you’ll need to use the ipwc.py CLI console distributed with PowerConsole. You may run it in your preferred debugger, but it’s not necessary under normal circumstances.
First and foremost, you may set the debug flag on your grammar by calling setDebug() method on nodes you want to analyze, so you’d get the debugging printout when these nodes are matched againt the command line. Typically calling it on nodes returned by _getGrammar() and _getCmdKeyword() is what you would want. However, if you would need to see the whole grammar parsing (for example to track down interferences between command grammars), you may use --debug-grammar command line option that will set the debug flag on complete grammar that PowerConsole uses.
If you’re using parse actions, you may trace their invocation with traceParseAction() decorator function from Pyparsing module.
When your grammar is basically right but you want to test how various alternatives are compiled into Python function calls, you may run the ipwc.py with --debug-calls command line option that will print out the preprocessed command line before it’s passed to the Python code compiler, so you could see the whole function call definition (including actual parameters) before it’s executed.
Command implementation itself happens in method execute() of your command class. However, to simplify command development, PowerConsole uses some convetions that you should follow. You can alway go your own route, but following these conventions would save you from typing and coding. There are also some constraints and conditions that your command implementation must satisfy to work properly.
Your command class must inherit from pwc.base.Command and define (override) next methods:
The pwc.base.Command class defines some useful attributes and methods. Namely:
The documentation for your command that is presented to end users by Help command is taken from doc string for this method. Alternatively you may implement the getHelp() method or use help provider object to document your command.
All commands have a name that’s used internally to look up the command class in command dictionary.
The command name is internally stored in _name attribute of your command class. The inherited __init__() method initializes it to your class name (lowercased) with ‘cmd’ prefix (if any) removed. You can change it to other value in your __init__ method after that.
The command name is also shown by builtin Help command in list of available commands (if your command is documented directly by command class), so your command name should match the keyword(s) used to invoke it.
Your command is executed as normal Python function in PowerConsole’s sandbox. Your may need to access objects defined in the sandbox namespace or execution context (for example when your command is executed from other function defined by user). The Command class has two useful methods for this purpose:
- _getUserNamespace()
- Returns dictionary with user objects (sandbox globals).
- _getContextLocals()
- Returns dictionary with local objects in execution context of your command or from specified frame context. To access caller’s locals (from user context) directly from your execute() method just call this method without parameter. However, if you’ll call this function from inner calls (for example from method called by your execute method), you have to specify the parameter value. The parameter specifies the number of frames to skip (i.e. level of nesting from execute to this function call).
For regular output, your command must use the display abstraction. For error and system reporting you may use methods write() and writeErr() inherited from Command class.
To obtain a display for your output, call the appropriate method on self.controler.ui_provider. It’s also a good practice to define the purpose string for your output. You should also document the display interfaces and purpose names your command(s) use, so tool developers could adapt their tools to allow customizations or specific handling to your output for end users.
PowerConsole supports builtin Help system that uses docstrings and special HelpProvider objects to print documentation for your commands and other topics you want to offer to users directly from the PowerConsole. If you want to list your command directly in list of supported commands, you must either write a docstring for execute() method implementing your command or define getHelp() method in your command class. Otherwise your command will not be listed. So if you don’t want to list your command (for example because you’ll document it via special topic handled by Help Provider), don’t create the docstring for the execute method.
To install command to the PowerConsole you must create the extension package object and implement method commands() that returns list of all command classes you want to add to PowerConsole engine. For example:
from pwc.base import *
class packageMyCommands(ExtensionPackage):
def __init__(self):
super(packageMyCommands,self).__init__('extension-package-name')
def commands(self):
return [cmdMyCommand,cmdMyOtherCommand]
If you want to install your command(s) to Standard Interactive Console or any application that supports PowerConsole extension packages, you have to register your package class as ‘powerconsole.package’ entry point in your setup.py. See Extension Package Specification for details. Otherwise you have to pass the instance of your package object to the Interpreter constructor.
HelpProvider is special object used by builtin Help command to display information for any topic or command you want to present to the PowerConsole users.
The simples way to install topics to the help system via Help Provider is to define your own class inherited from HelpProvider with string attributes or methods with names staring with HELP_PREFIX followed by name of the topic. While string attributes contain directly the help text for the topic, methods may simply return the help text or handle the help output by itself (for example Python Help Provider enters the Python builtin help system).
Simple Help Povider example:
class helpMyProvider(HelpProvider):
"""Help Provider for my PowerConsole commands and other topics"""
help_mylicense = """License text...
"""
help_sql = """Help for SQL support in my package.
"""
def help_mycomands(self):
...
return help_text
Installed topics: mylicense, sql, mycommands
If you want to create HelpProvider object with special handling, you have to implement (override) methods canHandle(), getTopics() and getHelp().
Next Help Provider implements access to builtin Python help system:
class helpPython(HelpProvider):
"""Help Provider that provides access to standard Python help system."""
def canHandle(self,topic):
"""Return True if 'topic' starts with 'python'."""
return topic.startswith('python')
def getTopics(self):
"""Return list of topics handled by provider."""
return ['python']
def getHelp(self, topic):
"""Return python help for 'topic'."""
topic = topic[6:].strip()
if topic != '':
self._help(topic)
else:
self._help()
Like commands, Help Providers are registered to PowerConsole by extension package object with method help_providers() that returns list of all help provider classes you want to add to PowerConsole engine. For example:
from pwc.base import *
class packageMyCommands(ExtensionPackage):
def __init__(self):
super(packageMyCommands,self).__init__('extension-package-name')
...
def help_providers(self):
return [myHelpProvider]
Some commands (yours or those created by other developers) may use Object Display interface to display objects to PowerConsole users. By default this interface displays string representation of the object (via str(<object>)), but PowerConsole supports more rich object rendering through user defined Object Renderers.
Object Renderer is an ordinary new-style class that can extend the Visitor Pattern used by PowerConsole or know how to handle object(s) of certain type(s).
The renderer must have the __init__() method that accepts at least one parameter: the display where the output should be rendered.
Note
Single renderer class can provide special printout for multiple object types.
If your command or other package implements objects that are accessible to console users, you may adopt the visitor pattern for them. Using the pattern would allow you or others to handle your object in specific way generally, not only special output, so it should be preferred over special visualisation by object type.
To add Visitor pattern to your object, simply add next method to it:
def acceptVisitor(self,visitor):
visitor.visit<class-name>(self)
The calling visitor must know how to handle your object, i.e. must have the corresponding visit{class-name}() method or know how to handle it in generic way, but it shouldn’t be the concern of your object. The Object Display used by PowerConsole will always know how to handle it. However, it doesn’t know how to handle it in specific way (does not have the visit{class-name}() method), unless you or someone else will not provide the renderer that implements it.
That’s it, your renderer class must implement the visit{class-name}() method on behalf of the display, and use the display’s interface(s) to write it out.
Warning
Your renderer class should always check wheter particular interface is implemented by display before using it!
If you want to provide special printout of objects that doesn’t and can’t support the Visitor Pattern, you can create a renderer that handle object by the type. To do so, simply add next method to it:
def handle_<type-name>(self,obj):
...
This method will be called by display for objects that are exactly of specified type (i.e. not for descendants or type-like objects that conform to the interface of specified type).
Here is the example renderer for list and dictionary objects that prettyprints them:
from pprint import pformat
class renderListDict(object):
def __init__(self, display):
self.display = display
def handle_list(self,obj):
self.display.writeLine(pformat(obj))
def handle_dict(self,obj):
self.display.writeLine(pformat(obj))
Like commands or help providers, your object renderer is registered to PowerConsole by extension package object with method object_renderers() that returns list of all object renderer classes you want to add to PowerConsole engine. For example:
from pwc.base import *
class packageMyCommands(ExtensionPackage):
def __init__(self):
super(packageMyCommands,self).__init__('extension-package-name')
...
def object_renderers(self):
return [myObjectRenderer]
Controller extender is a convenient way how to perform any action during PowerConsole initialization. For example when you want to add objects to user execution namespace, install attributes for shared use to the controller itself (all commands have access to it, so you can use it as common storage for data shared among commands) etc.
Extender is an ordinary new-style class that has the __init__() method or ordinary function that accepts at least one parameter: the Interpreter instance.
Like commands or help providers, your controller extender is registered to PowerConsole by extension package object with method controller_extensions() that returns list of all extender classes you want to add to PowerConsole engine. For example:
from pwc.base import *
class packageMyCommands(ExtensionPackage):
def __init__(self):
super(packageMyCommands,self).__init__('extension-package-name')
...
def controller_extensions(self):
return [myControllerExtension]
PowerConsole comes with one extensible command: SHOW (which you can use as example how to implement your own extensible commands). The purpose of this command is to provide a convenient way how to print out various information. Core grammar definition supports only expressions that are evaluated and rendered on display as lines of text (for strings) or objects. You can write your own classes that extend the grammar to display any information you want. For example database package may define extension to support topics ‘database’ and ‘tables’ to display information about connected database and available tables (via SHOW DATABASE and SHOW TABLES commands).
Extenstions to SHOW command are created like normal commands, with few specifics:
Here is the example extension that can show information about database procedures:
class showTest(ShowExtender):
def __init__(self,controller):
super(showTest,self).__init__(controller)
DBIDENT = (Word(alphas,alphanums+'_$').setParseAction(upcaseTokens) |
QuotedString('"',escQuote='"',unquoteResults=False)).setName('dbident')
self.keyProcedure = self._makeNameNode(Empty()) + (CaselessKeyword('procedure')
| CaselessKeyword('procedures')).setResultsName('category').setParseAction(upcaseTokens)
self.cmdShow = (self.keyProcedure + Optional(DBIDENT.setResultsName('objname')))
def _getGrammar(self):
return self.cmdShow
def _getCmdKeyword(self):
return self.keyProcedure
def execute(self,**kwargs):
"""PROCEDURES | PROCEDURE <name>
Shows list of procedures or detail about procedure.
"""
return 'Text to show...'