Safe Python Eval

Many of us want to use a safe simple scripting language to extend our Internet applications and I have seen many discussions on the Internet on how to implement Python so that it's eval function can't be exploited. The problem is, nobody can be sure that they've covered every scenario, because Python has become complex over time, with multiple ways to achieve the same objective.

My scenario is a typical one. I have a web based database application hosted on a remote server. Now I want it's users to access it's database to produce ad hoc reports, without compromising the database or anything else! I could do the typical thing of providing a screen where users can submit Select queries, but SQL by itself isn't enough to get a report that is useful except in the most simple scenarios. To begin with you can only work with one query result at a time. Reports frequently require multiple query results to be combined. Tables of course can be joined using SQL, but the result usually has lots of redundant data fields that must be omitted. A declarative language like SQL just can't do it alone. It needs to be combined with a procedural language. After all, that's how my web application works. The procedural capability of Python is combined with SQL to produce web pages that make sense to users.

After surveying the scripting languages out there, Python came in at number one, BUT, it has an eval function that can access the file system or carry out other security exploits. Vanilla Python isn't a sensible choice as an extension scripting language.

I have looked at FORTH in the past, ATLAST and Ficl were front runners, but they are traditional FORTHs with weak string manipulating capability. Strong string capability is what I need, because my database queries return as nested Python string lists. I came up with that format years ago before JSON became popular, using Python eval to parse the database query, returning a table that Python can iterate over using it's index functions.

Then I came across FORTH written in Python, a toy project for educational purposes. I realised it wasn't going to be fast, because FORTH is an interpreted language written in an interpreted language! What it did have going for it was very strong string capability and the ability to cherry pick the Python functions required, leaving everything else behind. Suddenly, eval wasn't a problem as I could overlay it with some checks to make sure eval could only eval a list. FORTH is a simple typeless stack based language with the entire environment comprising two stacks and a heap (Python lists), a dictionary (Python dictionary) and a heap pointer (Python integer). FORTH has no syntax! The language is just a stream of white space delimited tokens called words. Traditional FORTH has no error checking and crashes are common place. But having Python underneath FORTH  makes it uncrashable. Sure your FORTH code will frequently bomb, but the FORTH REPL (Read Evaluate Print Loop) just displays the Python error and awaits your next command.

I made some alterations to the Python code to get a FORTH that isn't quite FORTH anymore (as the saying goes 'when you've seen one FORTH - you've seen one FORTH')! Everything became a string, both the stack and the heap able to handle strings or lists of any size in addition to numbers. Actually Python is cleverer than that. Lists can store variables of any type, so my database lists stay as lists (under the hood, they're probably C arrays) which are passed by reference rather than value. Numbers stay as numbers and dates as dates unless they are cast explicitly, something I've brought over from Python. But apart from that the toy project has stayed largely untouched. All it can do is step, branch, loop, create variables, read variables and print to the console. But at just over 300 lines of Python, it's Turing complete and this is all you need to create a report. Already I have written a complex report utilising two tables that cannot be joined, the report output is styled HTML.

Perhaps the most interesting thing I've done is to create a persistant library of Forth definitions. The way I've done that is to store the FORTH environment after loading forth.py (the REPL) and forth.fr (the persistant library) as Python command line arguments. The working code FORTH file is then called manually in the REPL. Each time it is called the FORTH environment is restored to the finish of forth.fr. This means you can call the working code file over and over without having to restart the REPL.

But there's still a problem!  FORTH over Python might be safe, but a user could write a function that never returns. Endless loops are easy to create inadvertently in procedural languages. Code needs to be added to Isectd that checks how long a worker has been busy for. If it has been busy for longer than 20 seconds and has no clients attached then Isectd sends a kill command to the process which will automatically restart after it has been killed.

The working FORTH code is included in PocketClassmaker install file from the downloads area of my website.

Fox-Toolkit

I have been using Fox-Toolkit version 1.3.15 as my development environment for 15 years now. Prior to that I used Visual Basic 5, which I loved. Ever since moving to Fox-Toolkit I've been searching for a modern equivalent to VB5. I've never found it. Over and over, following these programming language journeys, I've concluded that Fox-Toolkit really is a brilliant piece of software engineering, all of it done by one man, Dutch born, Jeroen van der Zijp, beginning in 1997.

Here's why I think it's brilliant:

1. It's just a library written in C++, but Jeroen dislikes the STL and doesn't use it. The resulting language resembles C with inheritance rather than C++. Most of time I don't even use it like you would C or C++ as I have no need to concern myself with advanced concepts like inheritance, pointers and all the rest. Instead, I use it the way you used VB... without an IDE. In VB, you created forms and modules. Most of the code was written as event code associated with a widget and shared code was placed in modules. In Fox it's the same except that the form is just a standard *.cpp text file with Fox's window creation boilerplate and a module is a standard *.cpp text file without Fox's window creation boilerplate.

2. Because it's written in C++:

• it's runs extremely fast
• providing a project is set up correctly, compile times are quick, because most of the code is already compiled previously and only needs to be linked
• any external library written in C or C++ is easily integrated
• calls to the Windows API are a breeze
• I have multiple free compilers at my disposal
• the C syntax is terse and easy to read
• there are many code indenters available to tidy your code
• if I need to, I can program at a systems level, rather than at application level without having to move to another environment.

3. It doesn't use native widgets. Instead it emulates the look and feel of Windows 95, by using operating system primitives for drawing lines, filling spaces and obtaining mouse and keyboard input. This is the best part, because, Fox cannot be broken by a third party, since those primitive API calls are extremely unlikely to change. Also... I like Windows 95! Microsoft keeps tinkering with it's look and feel for marketing reasons, but in truth, nothing substantial has appeared since Windows 95 in terms of GUI usability.

4. It has a very powerful string class called FXString. I use this constantly. Most of my programming revolves around relational databases which I query using my own disconnected recordset object, written in Fox, which parses a JSON like string into linked lists of FXString. Very simple, very fast and rock solid performance.

5. The window geometry manager avoids fixed window sizes, so Fox applications resize themselves sanely. VB didn't come close in this regard.

6. Fox comes with it's own code editor called Adie. Of course, any code editor is viable. Notepad++ is popular. But I like Adie because it's so simple.

7. The widgets collection is huge and with a bit of digging in the code you discover things are available that you don't expect. For example, recently I found that the FXTable widget lets you copy it's contents to the clipboard in tab delimited format. Spreadsheets parse this into columns and rows directly without the usual dialog palaver that comes with a CSV file. Other time savers include INI file creation which comes baked into Fox allowing window positions to be saved on exiting your application.

8.  Being open source with a liberal licence, you have access to all the source code which is reasonably easy to follow, so you can see exactly how things work. The situation has yet to arise, but if a widget doesn't have a feature I want, I can inherit a new widget from it that does exactly what I want.

9. Cross-platform development is the goal of many tools, Fox included. But, I have come to see that real world application programming is about being able to call into an operating system's API to achieve outcomes that aren't clunky workarounds. For instance, I needed to convert rtf to pdf sanely and programmatically. The best way of doing this was to call LibreOffice in headless mode, but that resulted in console windows appearing. I found the appropriate Windows API call to suppress the console windows and substituted it for the normal system() call. While I could have done something similar in other languages it would have been much more difficult to implement. With Fox it was just a bit of cutting and pasting.

Perhaps the biggest benefit of all from using Fox, is that you cannot code yourself into a corner.  So many times you start to use a tool only to find that there is some aspect of it that constrains your development. In 15 years of developing with Fox that is yet to happen. I've produced desktop applications and web sites, just using Fox-Toolkit. Check it out.

Disconnected Recordsets

If you Google the term 'disconnected recordset' the results returned refer to old style Microsoft ADO recordsets which are able to use database cursors, but Microsoft has stopped using that terminology, because their newer ADO.NET recordsets are always disconnected. Database cursors have dropped out of favour because they maintain state on a database, which isn't scalable, particularly on web servers.

Reading some of Microsoft's documentation today surprised me, because ADO.NET's approach to database access is nearly identical to my own hand-crafted disconnected Recordset object, which I came up with nearly 20 years ago, roughly the same time ADO.NET 1.0 was released.

What does my Recordset object do?

1. It parses a tokenised string from the database driver or from a file into a double linked list which is then referred to as a recordset.
2. It contains methods to iterate forwards and backwards through a recordset.
3. It contains methods to retrieve the contents of a field by name or position from the current row in a recordset.
4. It contains date and time methods.
5. It can sort a recordset.
6. It can emit a recordset as a tokenised string.
7. It can write a recordset back to the database in one round trip.
8. All data access is carried out under transaction management.
9. Additional methods can be added to the object as needed, without impacting earlier code.

Why did Microsoft move to disconnected recordsets from cursors?  I have found the following benefits:

1. Multiple recordset objects share the same database connection which is opened automatically when the first query is run.
2. A recordset object can act as a record datatype without using a database.
3. The recordset object hides a lot of the repetitive insert, update, delete queries that come with SQL. SQL queries are still used a lot though. Without in depth knowledge of SQL, you can't use relational databases effectively.
4. As the recordset is a tokenised string it can be written to disk and retrieved again.
5. Transaction management can span multiple recordset objects.

OK, what's this continual banging on about tokenised strings!  It turns out that these are the key feature that makes the whole disconnected recordset thing work.  Here's a tokenised string:

[[{'name':'ID','type':'i','null':0,'width':4},{'name':'DESCRIPTION','type':'s','null':1,'width':30},{'name':'REORDER','type':'i','null':0,'width':4}],
[[          5,'Moved',          5],
[         10,'Critical',         10],
[         20,'Urgent',         20],
[         30,'Normal',         30],
[         40,'Wish',         40]]]

Through a historical accident, it so happens that this tokenised string is also a Python list. Perhaps if I was to reimplement it today I would use JSON which is more widely used. Nevertheless, this format works well. It can be evaluated directly using Python and I have written a parsing routine in C++ that converts it to a double linked list in a single parse.

JSON can exceed two dimensions and use schemas, but the column and row format is what makes my tokenised string so efficient to parse and to manipulate once parsed. Columns and rows are easy to understand and column numbers rather than column names can be used for faster data retrieval. Not that this makes much difference, since a linked list is very fast to iterate over, so I usually use column names instead of column numbers to maintain code legibility.

Looking at the string you can observe that it is restricted to columns and rows, but it isn't a flat file. Instead, this format lies somewhere between a flat file format like dBase which has fixed column widths and is very fast to query using file offsets and CSV files which don't have quite enough information to be useful as a recordset.

Here's another tokenized string:

[[{'name':'ID','type':'i','null':0,'width':4},{'name':'MSGID','type':'i','null':0,'width':4},{'name':'FKISSUES','type':'i','null':0,'width':4},{'name':'FKPEOPLE','type':'i','null':0,'width':4},{'name':'BODY','type':'s','null':1,'width':32765},{'name':'STATUS','type':'i','null':0,'width':4},{'name':'LAST_ACTIVITY','type':'d','null':0,'width':8}],
[[          4,          4,          3,          6,'What would we talk about?',         30,'2018-06-02 15:10:07.0000'],
[          6,          6,          3,          6,'I need to keep going with this software.',         30,'2018-06-07 21:01:40.0000'],
[          7,          7,          3,          6,'Another note...',         30,'2018-06-19 11:16:40.0000']]]

Notice that the field descriptors contain the field type. This is vital for sensible parsing of the recordset. Type inference which is what you have with CSV files can only get you so far.  Knowing the data type of the field beforehand allows for clever data massaging e.g. my recordset object has an optional flag for elapsed time which represents dates as duration from now rather than the raw format you see here.

There's a few things that aren't so great with disconnected recordsets:

1. They can't store binary data. I get around this by converting to Base64 which is slow. Over time, I've come to realise that binary data is much easier to use if it is kept out of a database anyway.
2. How do you deal with deeply nested data? Relational databases don't store data hierarchically.  Instead they use relations between tables. It is possible to store data hierarchically using table relations, although a little tricky (a whole blog post could be devoted to that subject). Recently, I have found the need to store data hierarchically, but I do that using the database tables themselves, so it has no effect on my recordsets.
3. What about shallow nested data? I use Base64 encoding to store recordsets inside recordsets, since none of the Base64 characters match the recordset's delimiters, but I've only encountered one instance where shallow nesting was needed.
4. The parsing routines must escape delimiting characters. I use octals to avoid the escaped representation containing letters which hexadecimal requires.

Technical Details 

• disconnected Recordset client (data view)
• C++ using Fox-Toolkit which has an excellent string class that I use heavily in my projects.
• eval in Python is all you need.
• disconnected Recordset driver (data adapter)
• native driver written in C for the FirebirdSQL relational database.
• disconnected Middleware (connection)
• both the client and server depend on IsectMP to work.  IsectMP is a tiny RPC multi-plexer written in C that I maintain.  It contains the FirebirdSQL driver, but not the C++ client.

What's wrong with CGI?

The accepted answer is that CGI is an outdated concept that no longer has a place in modern web applications.  I'm not so sure.  Let's look at how CGI works.

1.  The user requests a page.
2.  The web server starts up a new process, serves the content and the process dies.
3.  If the page is doing anything interesting that invariably means the CGI process must open a database connection and destroy it again.

In the past, operating systems weren't smart and starting a new process mean't reading the binary image into memory from disk. That's not true now, particularly for Windows (I'm a Windows developer, so I had better stick with what I know); it caches all recently read files in memory. So, if your CGI file is a compiled executable then the overhead of CGI has gone. If your CGI uses a scripting language such as Python, Perl, Tcl etc then their virtual machines must parse your script every time which will slow things down, but not by as much as you'd think, since both the virtual machine and the script are cached in memory and in Python's case at least, the script gets compiled to byte code on the first parse and thereafter runs much quicker.

Opening and destroying database connections is expensive, mainly because the database server must authenticate your connection. But if you insert some middleware in between the database and your CGI file and pool your database connections, the authentication overhead is removed. Ensuring that your database recordsets are disconnected recordsets also makes things run much faster.

So to recap, if you have CGI executables using disconnected recordsets from pooled database connections what's wrong with CGI?  Let's count the advantages:

1.  The CGI executables are tiny.  I have a single EXE for each page, all of them sharing a 2 Mb dll.  The EXE's never exceed 200 Kb. They compile in a flash.
2.  Because each CGI is stand-alone, I can keep the web site running, while just working on one aspect of it. And if a CGI should crash in production, the whole site isn't brought down, just that page.
3.  The site runs exceptionally fast, because the CGI files only do the bare minimum to create a page.  No lumbering template libraries here. In fact, no template libraries at all!  HTTP, when you remove all the towering library abstractions built on top of it, is actually very basic. Just GET and POST calls passing around variables, HTML Forms that collect the data into those variables and cookies to maintain state. It also helps to have a little bit of Javascript sprinkled about so that your Forms don't need to return to the server for every action the user takes. A typical CGI file begins by collecting any variables sent to it, reading cookies and opening a number of disconnected database recordsets.  The variables and cookies are transformed and saved as appropriate and the file finishes by writing out a new page. In my files less than a third of the code is devoted to writing the new page. Most of it revolves around updating the database.
4.  Secure. You can't have a scripting attack, if all the code is compiled into an EXE.

The disadvantages?

1.  My code uses combination of C and C++ supplied in three libraries. First, cgihtml-1.69, a minute CGI library written in C, that I scrounged off the web and which was last updated in 1998. Second, fox-1.3.15 written in C++ and released in 2002 (Fox-Toolkit is still current. The latest release is 1.7.61, but why update when the existing functionality surpasses anything I'll ever need?) and third, isectMP-1.0.3, a database pooling library written in C which I maintain. Fox-Toolkit comes with an excellent String manipulation class which I use constantly, but sometimes I yearn for the simple syntax of Python. That yearning soon disappears when I have to interface with a third party library or interact with Windows directly! And what about Python deployment? Let's not go there!

2.  Writing CGI's in C and C++, just isn't fashionable.  I constantly have to remind myself that it's results that matter, not that I should be using the latest language du jour.

And so, I find myself asking that question again, what is the problem with CGI?  Answer:  Nothing!

PC based Lesson Planning Software Shootout

It seems the writing is on the wall for PC based software.  Just three years ago there were at least five actively maintained lesson planning packages available for Windows users.  Now there are just two, my own PocketClassmaker and Jeff Hellman's Planbook 4. After looking at his Facebook page I'm not even sure that Jeff is still in business, since there are users requesting support and they have not heard from him since July 2015. To be fair, I haven't posted a software update since June 2015, so perhaps we're in the same boat!

Most teachers have probably moved to Web based solutions by now and they do have their advantages, but if you like me, prefer to have control over your own data and aren't constrained by school policy to use their mandated Web solution, then perhaps PC based software might still work for you.

I compared both packages exhaustively for usability, but omitted testing Planbook's PlanbookConnect which is some kind of Web based synchronisation service, since PocketClassmaker can already do that using Google Drive and I didn't want to sign up to PlanbookConnect.

If you can't be bothered to look through my comparison table below here's the Executive Summary:

Both packages cover similar territory, but from different angles which results in a different user experience:

1. Planbook is optimised for rotating lesson plans (create a rotating schedule by subject first and then fill in the detail later) for a single user and cannot be used as a normal calendar. It is relatively painless to get going, but has a number of features which I found very frustrating. These include entering lesson times which is fiddly at best, the startup screen position cannot be saved and  perhaps the biggest frustration, to save your work you must use the Ctrl S key combination constantly or risk losing your planning. There is also a bug that crashes the program when trying to access the Lesson Library. To allow the software to be used on several machines Planbook has a synchronisation function called PlanbookConnect which I didn't test.
2. PocketClassmaker is optimised for ad hoc planning (plan as you go by copying existing lessons forward and amending them to suit) with multiple users and thus can be used for any task that is calendar related, but, the learning curve for installing and using it is steeper than for Planbook. PocketClassmaker is a portable app and is designed to be installed on a USB stick.

 

	PlanBook	PocketClassmaker	Comments
Website	www.hellmansoft.com	www.creditscore.co.nz
Windows	64 bit Windows 10 Home	64 bit Windows 10 Home	PlanBook versions exist for Mac, Windows XP and Ipad. Classmaker only works on Windows XP and up.
Screen size	1366 * 768 on 14 inch screen	1366 * 768 on 14 inch screen
Version	4.0.51	2.0.6	Date downloaded 07 May 2016
Cost	USD30 for one user	Free for one user
Installation	Windows MSI. Only allows installation to C:\Program Files (x86)\Hellmansoft\PlanBook	Extract Zip archive to any location	Classmaker is a portable app.
Google Drive	No	Yes	Only one instance can be active at any time for Google Drive replication to work.
Invoking	Normal menu item	Two batch files located in the Scripts directory.
Multiple users	No	Yes
First invocation	Wizard	Blank screen	Classmaker comes with an example database.
Remembers window positions	No	Yes
Default dates	September 2013 to June 2014	Have to create a new user and a default non-contact lesson plan before the screen displays. Defaults to today’s date.
Subjects	Created using Wizard	Defined at Syndicate Leader level
Time entry	Fiddly input fields that require am/pm	Uses 24 hour clock	PlanBook's approach of using option boxes is disastrous from an inputting perspective. The a.m / p.m selection often fails to take correctly and the time reverts to the initial setting.
Date entry	A lesson can only belong to one day	Lessons can cross multiple days using the same start and end times	It is almost impossible to use PlanBook for add hoc planning as white space is not allowed
Fast Date Entry	Not needed because lessons are always prepopulated using some kind of rotating schedule	Just enter the day for the current month	PlanBook only works with rotating schedules
Database type	Individual files for user defined periods	One database file for all users and forever
Autosave to Database	No	Yes, but required to explicitly add, edit or delete a record	Losing your work after some time doing data entry in PlanBook is VERY frustrating.
Long Term Plan	No. Month display shows Subject name only.	Yes. Term display shows Subject and Unit Name.
Unit Plan	Optional. Used for printing and exporting / importing lessons.	Mandatory.
Permanent Lesson Storage	No. PlanBook crashes!	Yes	PlanBook developer’s main focus appears to be on Apple Mac’s.
Weekly Report	Attempts to include detail	Just prints lesson names
Daily Report	Ugly cross page splits. Includes empty lessons.	Nice table display
Future and Past Lessons	Can’t tell if you’ve created future lessons	Long term display tells you immediately if you have done future planning
Attachments	Attached to Lesson Plans using a link	Attached to Unit Plans and stored in database file
Website links in lesson plans	Yes	No	In Classmaker would have to use HTML tags eg. <a href=http://www.stuff.co.nz>http://www.stuff.co.nz</a> and print to HTML
Report Types	Daily, Weekly, Unit	Daily, Weekly, Term, Unit
Report Formats	Print only	Print, Postscript, HTML	In Classmaker HTML reports are similar to CSV files and can be imported into MS Word or formatted using CSS for display in a Web Browser
Database Export	Yes, but empty days are exported too. Assumed that database will be published to a web site between dates.	Yes. All date and time relationships have been removed. Just the data and file attachments are exported. Assumed data will be globally searched including file attachments.
Search	All fields in database, but not attachments.	Only lesson titles.	Both packages require database export to search attachments.
Lesson Types	Unit	Unit, Routine and Non-Contact
Schedule Types	Same subjects repeated daily, same subjects repeated weekly, rotating schedules up to 14 days	Ad hoc, subjects repeat daily, subjects repeat weekly, rotating schedules up to 30 days
Lesson Movements	Bump, Yank, Lock	+Day, +Week, Bump, Shove, Up, Down
Unit Export	Attachments aren't exported only links	Multiple units can be exported as a batch including attachments
Rotating Lessons	Set up lessons beforehand and then use directly	Use a placeholder unit that must be copied forward again and again
Field Input	On failure reverts to initial figure	On failure input figure retained but not written to database

Does your office have Issues?

1. If your office responds to repeat customer requests that require various staff in your office to carry out a number of time consuming tasks sequentially,  you have issues.
2. If on the other hand your office generates and disseminates information to multiple contacts, you don't have issues.

Many offices have issues with their issues, primarily because they are a type 1 office trying to use type 2 office tools!  A type 1 office needs to have a central email repository within which to store their issues.  Instead, many type 1 offices use standard email clients which are designed for type 2 offices.

As an electrician my office is a type 1 office.  Retailers and marketers have type 2 offices.  For my centralised email repository, I use Roundup Issue Tracker.

In a corporate environment Roundup does three things particularly well:

1. Acts as a email repository that can be easily searched by all staff.  Issues don't get lost in Roundup, because they can never be deleted, just retired.
2. Makes an excellent workflow tool that tracks issues and amends their status as they move through their lifecycle in the organisation.
3. Ensures that customers can easily communicate with staff, but in such a way that inappropriate information never crosses the boundary between the small office and the outside world.

But, to do these things I had to make some changes to Roundup, because Roundup's original target audience is software developers not corporate office environments. To make Roundup a happy camper in your typical small office environment, my version of Roundup now looks like this:

• I distinguish between internal users and external users, since external users only interact with Roundup from their email client while internal users favour its Web interface.  For this distinction we have four types of user:

1. Anonymous - Roundup does not know their email address
2. User - External user
3. Staff - Internal user
4. Admin - Administrator

• The Staff role is a new role that I created.  It is appended to the other roles to obtain the required behaviour eg. (Admin, Staff) or (User, Staff)

• I send out HTML emails which include corporate signatures to external users.  Internal emails remain as plain text.

• By default Roundup only sends emails internally to Staff. To send an email to an external client, Staff have to set a flag on their message.

• If Roundup is to act as an effective repository, it's database must not be polluted with spam. To prevent spam I wrote the following business rules:
• If an email sender is Anonymous their email is bounced to an Administrator.
• If a User (not Staff) is known to Roundup, but they are attempting to create a new issue, their email is bounced to an Administrator.
• In both cases, if the Administrator determines that the email is not spam, they forward it back into Roundup, where it appears as a new issue.

Why I love Units of Work

Not many lesson planning packages use Unit/s of Work (UoW) to organise their lessons. Classmaker uses them for everything! What is a UoW? A UoW is a container that holds a lesson collection. The lesson collection comprises individual lessons and supporting files. UoW are similar to tags, but unlike tags, a lesson cannot exist without belonging to a UoW and a lesson can only belong to one UoW.  Conversely, multiple tags can be assigned to an individual lesson and by default a lesson has no tags.

This graphic shows how the major components of Classmaker relate to each other using a Venn diagram.

At face value, UoW seem too restrictive..., and tags would appear to be a much better bet, but as I will show, this is not the case.

Pros

1.  Because a lesson must belong to a UoW, a UoW can be much more descriptive than a tag and combine operational information with data. For instance, in Classmaker, the UoW has a title, a short description, the lesson type (non-contact, routine, unit plan, rotating), whether the UoW has just been imported or if is marked for export and a list of all the supporting files attached to it. The subject of the lesson is listed in the Long Term Plan (a superset UoW) along with whether you want to display it's lessons on the Weekly Calendar or not. All of this is information that you don't want to have attached directly to individual lesson plans as some of it is operational information rather than data. Even the data may not be required on some lesson reports. But this is information that you want to be able to refer to easily if you need to, which is discussed next in 2.

2.  UoW's are reciprocal.  By this I mean that if you click on a UoW you can easily view all the child records that belong to it.  But this also works the other way around.  If you click on a child record you instantly know the parent UoW that it belongs to as well. Classmaker uses reciprocity extensively in it's user interface. Whenever you click on a lesson plan on the Weekly Calendar, the lesson detail brings up the UoW that lesson plan belongs to, all the other related lessons in the same UoW, the Long Term Plan the UoW belongs to and all the other related UoW in that Long Term Plan. This visual overview of your planning, Classmaker retrieves for you automatically, every time you click on a lesson plan.

3.  It is much easier to work with groups of records rather than one record at a time. UoW's facilitate this since a lesson can only belong to one UoW. When you delete a UoW, everything inside it goes too. Conversely, to create a clone of a UoW, export and import the entire UoW (1, 1a or 1b) via the hard disk.  The UoW keeps track of whether it is flagged to be exported, is waiting to be imported from disk or has recently been imported, rather than having to concern yourself with the state of the individual lessons involved.

4.  Batch jobs have been used in computing for a long time. The advantage of using batches is that small groups of records can be manipulated together, without exposing the database to the risk of all it's structure being changed beyond reversal by a single SQL statement. UoW's are analogous to a batch job. They can be cloned, deleted and their lesson plan dates changed using Shove. If the result is not what is expected, the cloned UoW is deleted.  Classmaker users will discover that they often use UoW's clone, delete and edit functionality to create new records from existing records as it is much easier to clone, delete and edit records than it is to create them from scratch, particularly if you use Rotating Schedules.

5.  A Classmaker user will clone and edit whenever they can.  For instance, say you need to create a new record in the same time slot as today, but for tomorrow in a different subject.  In all the other packages I've tried, your only solution would be to create a new record from scratch.  In Classmaker, you take today's record, push it forward one day (+Day) and then add it into a different Long Term Plan and Unit Plan (2c.), just four mouse-clicks with no typing of dates and times. This works due to UoW's reciprocity, mentioned above in 2.

6.  Hierarchical data structures are simple to export to disk as that is how the file system works. The parent child relationship implicit in UoW mean their existing structure can be written to disk intact. All other packages I've tried, export their lesson plans as flat files destroying their user interface relationships in the process (that's if they've even tried to export their lessons. Many vendors don't bother).

Of course, UoW's aren't all beer and skittles!

 

Cons

1.  A lesson cannot belong to multiple UoW.  With tags this is a no-brainer and their most useful attribute.
2.  You do not know at a glance all the different UoW you've created, but it's easy to see all your tags.
3.  The tags associated with a lesson can easily be changed at any time. It's more difficult to move a lesson plan from one UoW to another.

 

Summary

I prefer UoW to tags because their hierarchical structure readily translates to data hiding, reciprocity in user interfaces, easy exporting and importing, batching of work, cloning and editing records instead of creating new ones and database exporting to disk without destroying parent child relationships.