This essay is directed to a particular but rather large set of Mathematica users. Mathematica is such an expansive and powerful application that there are many usage modes. This essay is directed to individuals who are using Mathematica to develop a base of knowledge and a set of capabilities centered on some subject matter. This might range from students to advanced researchers.
The problem is how to organize the material so work is accumulated and preserved. You may write numerous notebooks and routines within various notebooks. If these are just scattered about it’s difficult to build capability. Mathematica provides a beautiful framework for developing, organizing and preserving work.
This framework is called an Application and there is a specific place to put it: your $UserBaseDirectory/Applications folder. This is the place that WRI has provided to put your personal work and any other place (except $BaseDirectory) will have serious shortcomings. Mathematica automatically searches this location for various resources you may have provided. If you send your Application to a colleague and he places it in his own private Applications folder he will have no difficulty in accessing the resources. These include notebooks, packages, palettes, style sheets and documentation.
You might say that you are only a beginner, or that you are just starting a project, or that you don’t yet have packages, palettes, style sheets or documentation – don’t even know how to do them. You might argue that your work isn’t important enough to be a Mathematica Application (Then do something more important?) All of these are poor excuses. Your work is important. You may in time add extra features and you are in the right place to do it – simply by adding on.
So you might start out with the following folder structure for an Application that deals with MySubjectMatter:
$UserBaseDirectory/Applications/ MySubjectMatter PublicFolders PrivateFolders
The PrivateFolders folder would contain all your development notebooks that you don’t want other people to see. Then if you want to send your work to a colleague you can simply zip the MySubjectMatter folder, do a single delete of the PrivateFolders, and send it off. The folder structure above doesn’t yet have any of the usual folders or files in an Application but at least you have a definite place for work on MySubjectMatter and a way to share the work.
I encourage the writing of finished or literate notebooks with textual explanation and intended to be read by other people. We might call these literate notebooks as opposed to scratchpads. Among other things, this requires understanding how to enter Section headers and Text cells. Simply put: if you can’t explain what you’re doing you probably don’t understand it well enough and you may have a difficult time integrating your work into the larger world.
When working on a notebook Section, I often find that it’s not working out or that I don’t understand the material well enough, or that a calculation is clumsy or too particular. I often mark the Section “Try 1”, copy it and work on “Try 2”. Sometimes I get up to five or six “Tries” before I’m happy and then delete the earlier efforts.
One advantage of this approach is that you end up with notebook material that you can communicate to others.
Generating Package Routines
Mathematica has extremely powerful mathematical routines and year by year becomes more convenient in its usage. Yet, science and mathematics are so vast that it is inevitable that you will eventually want to write your own specialized routines and put them into packages. This builds up capability and allows you to distribute that capability to colleagues.
So how do you generate useful routines? Make a survey of your subject matter, write a list of all the routines you may need and then proceed to program them? This will probably not work well, especially if you don’t have a lot of experience with Mathematica or are still learning or exploring the subject matter. The following is a better method: Write tutorial or expository notebooks for your subject matter. Find out what Mathematica will do on its own. Eventually you will come across calculations that require many steps with detailed processing and are required to perform with varied sets of data. These are candidates for package routines. Good routines require usage and experience as well as theoretical considerations. Providing that usage is a key part of mastering the subject matter.
When I discover such a routine I usually first develop it within the Section it arose. Then I move it to a Routines Section at the top of the notebook and make it an initialization cell. I call this Package Purgatory. I like to use and test the routine for awhile. I may even copy it to other notebooks where it is used. I often find that I have to do further refinements – or even complete redesign.
What does a developed routine look like? It should have the following items:
- A usage message
- Possible error messages
- A SyntaxInformation statement
- An Options definition if the routine has options
- An Attributes statement if the routine has any special Attributes
- The routine itself
One should write the usage message before writing the routine. If it’s difficult to explain the usage then the concept of the routine is probably ill-conceived.
Once you have all these things and a routine has undergone testing with an adequate variety of data it is ready for Package Heaven.
Adding a Package
When you have a collection of generally useful routines you can add a package containing them to the MySubjectMatter Application. Initially, Mathematica had the following mechanism. Write a regular Mathematica notebook with the routines as initialization cells and set the notebook (in Options) to automatically generate a package.m file when saved. There is a better way now, which is to do away with the package.nb notebook and move routines directly the package.m file.
Package files allow Titles, Sections and Text cells just as regular notebooks. So it’s possible to structure them in an organized manner. The code in package files is put in Code cells rather than Input cells. You can switch between them by selecting the cell bracket and using the right click context menu or using Alt-8 or Alt-9. One can save an old version of a routine by switching it to an Input cell and having the active version in a Code cell.
It is useful to switch to Input cells when editing usage statements. But switching to Input cells for routine cells is more problematic. The typing behavior changes and converting to StandardForm can introduce undesirable changes. So here it’s better to edit the Code cells directly. One can change the notebook magnification to make them easier to read. It’s also possible to “pretty format” the code by introducing line returns and spaces to make it easier to read.
The package file needs a name. Let’s make it different than the Application name, say SubjectMatterCode.m. The BeginPackage statement needs to specify the path to the package starting from the Application folder so it generally has a binary name:
As standard practice, there will also be an init.m file in a Kernel folder. It will contain the statement, which performs the actual loading of the code.
There could be additional packages, in which case there would be additional Get statements in the init.m file. WRI has then arranged it so that the entire Application can be loaded with the simple statement:
Mathematica will automatically go to the init.m file and execute the statements there.
The folder structure will now look as follows:
$UserBaseDirectory/Applications/ MySubjectMatter FrontEnd/Kernel/init.m SubjectMatterCode.m PublicFolders ColleagueFolders PrivateFolders
You now have a working Application with a package. If you send it to a colleague and he unzips it directly into his private Applications file he will have access to all of your code and can load it from anywhere in his Mathematica files. The user would not have to set a Directory path. He could even add a Folder for his own notebooks and, since you don’t have that folder in your distribution, installing an update would not disturb his folder.
Style and Style Sheets
You may want to have your own style sheet for your Application. Mathematica has many available style sheets but every one has something quirky about it. There are so many possible variations that designing style sheets might be considered an art in itself. Here I will just mention some items that I consider important in a style sheet and general use of Mathematica, then explain how to create style sheets and incorporate them in the Application.
I prefer a “classic” style without adornment. Anything extra, such as text color or frames, has to be decoded by the reader who probably won’t understand your coding. I like to think of a Mathematica notebook as a blank sheet of paper that I am writing on. Isn’t that the way you do mathematics or sketch out your ideas? I don’t want material inserted in my notebook that I didn’t write or generate as computed output. Obtaining a style of working that works for you requires a combination of preference options and style sheet entries. My preference (set in the Interface tab of Preferences) is to turn off autocompletion, floating elements and especially Show Suggestions. I don’t like autocompletion continually flashing choices before me when most the time I know perfectly well what I want. Nevertheless manual autocompletion is useful, especially when typing long symbol names. This can be done, quite fast, with Ctrl+K and Ctrl+Shift+K. I find the Suggestions Bar less than useful. The suggestions might be useful to beginners but otherwise are quite shallow. To me they are another thing that has to be brushed out of the way.
Similarly, there’s the CellInsertionPointCell. This is an icon that lets you choose between an Input cell, a Text cell, free-form input or Wolfram|Alpha. But there are easier ways to obtain these choices without cluttering the notebook with an additional decision point. You can obtain Wolfram|Alpha, which has useful but rare usage, with “==”. I’ve programed the two extra keys on my Logitech mouse to enter Shift+Enter and Alt+7, the codes for evaluating an Input cell and for starting a Text cell. So the CellInsertionPointCell adds nothing but clutter. It can be gotten rid of by adding a Notebook option in a style sheet, or by looking it up alphabetically in the Option Inspector and changing it to:
CellInsertionPointCell -> None
I realize many will find benefit from these Mathematica features. Then by all means use what is helpful. Nevertheless, my mantra is: Clear clutter from the path, then proceed with the math.
Returning to style sheets proper, the next task is to create a folder in your Application for your style sheet(s). In my opinion WRI has not yet provided a simple procedure to add style sheets to an Application. Most style sheets will be associated with Applications and they should be in the Application. Therefore I’ve done the initial work for you. There are two files that are most useful in designing a style sheet: A StylesheetPreview.nb and an EmptyStylesheet.nb style notebook. You can download these two files at the Mathematica Style Notebooks location at Dropbox. The StylesheetPreview notebook contains the major notebook cell types. As downloaded it is in the Default style. You can set it to the style sheet you’re developing and immediately see the effects of changes. The EmptyStylesheet notebook contains a style sheet essentially based on the Default style. Open it and then use SaveAs (Save will not work) to save it under a new name, say MySubjectStyle1.nb. You might have more than one style sheet but we’ll just start with one. You will have to quit and restart Mathematica for it to recognize the new style sheet.
Your Application folder structure will now look as follows:
$UserBaseDirectory/Applications/ MySubjectMatter FrontEnd/ Kernel/init.m StyleSheets/MySubjectMatter/ StylesheetPreview.nb EmptyStylesheet.nb MySubjectStyle1.nb SubjectMatterCode.m PublicFolders PrivateFolders
Notice the extra folder, MySubjectMatter, inserted after Stylesheets. This is a courteous feature to use. It lumps all of your style sheets under one heading in the Stylesheet section of the Mathematica Format menu. So your Application will make only one main entry on a user’s menu. Even if you have only one stylesheet it helps to identify the Application that uses that style sheet. (The style sheet notebooks have a button Install Stylesheet. In Mathematica 10.2 it will only install in the System files location. Don’t use it; we already have a home for the style sheet. Mathematica will have no trouble finding it in the Application.)
The following are what I consider desirable features for a style sheet:
- A not too large gradation in font size between Title cells and Text cells.
- Input, Output and Text cells should have the same font size. That is what published papers use.
- Input cells can use a very light background color to distinguish them from Output cells. Otherwise, Input, Output and Text cells should have no extra adornment. Then you can close the Input cell and the Text and Output cells will read like a published paper.
- Section and the various subsection cells should have GroupOpeners on them. Input/Output groups should not.
- There should be sparse or no use of color, except for the background of Input cells or Title and various Subtitle cells. Black is the best choice for font colors.
The following is a straightforward procedure for creating style sheets. Essentially we take an existing Default style sheet and modify it.
- Open the StylesheetPreview notebook and the MySubjectStyle1 style notebook.
- From the Mathematica Format menu choose Stylesheet, MySubjectMatter, MySubjectStyle1. This puts the preview notebook in the new style sheet, which at this point is essentially the Default style.
- You can change a style by using the Choose a style drop-down menu or typing a style name. Selecting a style will paste its style definition into the style sheet notebook, which you can then edit.
- Alternatively, click on the Default.nb notebook link at the top of the style sheet. This brings up the Default.nb style definitions notebook in a new window . This style sheet has the advantage of showing the major notebook styles grouped under Section headings. You don’t want to edit this standard style sheet, but you can explore the contents and you can copy (with Ctrl+C) and paste styles or entire sections into your private style sheet. Or you could use the Section headings to organize your private style sheet.
- You can even dig one level deeper by clicking and opening the Core.nb at the top of the Default style notebook. Here you will find styles for a large assortment of low level styles that you will mostly not want to change. Some styles you may want to change, such as using a darker color in the local definition for “Inactive”, or a larger font size in the style for “PrintUsage”. Copy the cell definitions and paste them into your private style sheet notebook.
- Once you have the style definition cells in your private style sheet notebook you can edit them either from the Format menu or by opening up the underlying style expression (Shift+Ctrl+E) and typing in the desired options. The styles are defined principally by options. I rather prefer the latter method because it is more precise and general. You may have to look at the Default or Core definitions to see what all the possible options are. Remember that the style sheets are cascading so you only have to put in changes.
- When you make a change in the style sheet it will immediately appear in the StylesheetPreview notebook.
- Once you have added all the desired style sheet modifications, use SaveAs to save the style sheet back in the FrontEnd/Stylesheets/MySubjectMatter folder. Mathematica should automatically start at that folder, but often with new versions WRI loses that feature until users complain. So it pays to double check. You can’t use Save and it doesn’t give you a warning that it didn’t Save so be careful here.
The style sheet is now installed and anyone who has your Application can use it for any of their notebooks. And, of course, any expository or tutorial notebooks you distributed with the Application would probably use your style sheet.
Palettes are problematic. They should only arise when they serve a purpose. They consume expensive real estate on the user’s screen and are subject to palette clutter. The mantra for palettes is: maximize the functionality, minimize the space. The opposite of that is turning a palette into a billboard with unneeded items.
It’s difficult to design a general palette for Mathematica because Mathematica is just too vast. However, it’s possible to design reasonable palettes for smaller functionally related sections of Mathematica and similarly for Applications. The ideas is to provide a quick overview of what routines are available, links to their documentation pages (if they have them) and template pastes for the routines. Two such palettes, one for the Mathematica Geometry routines and one for the Tensor routines are included in the Mathematica Style Notebooks Dropbox folder. If you have Mathematica you could download these palettes and put them into your private Mathematica/System Files/FrontEnd/Palettes folder. Then bring up the Geometry Palette.
Two constructions that are especially useful in palette design are drop-down ActionMenus and Tooltips. On the Geometry palette routines are grouped under various headings. Hovering the mouse over one of the headings provides a tooltip listing the routines grouped in that heading. There are 83 routines covered in the palette and the user can identify them simply by hovering the mouse over the various groups. Next to each heading is a “?” button. These provide Action menus for linking to the documentation Help page for each routine in the group. There is also a “? Guides” Action menu that provides links to various WRI Documentation Guide pages concerned with Geometry, and also a WRI tutorial video. Finally the headings provide Action menus for pasting Geometry routine templates into a notebook.
If there are many palettes in an Application then palette clutter becomes a concern. It can be partly alleviated by treating most of the palettes as temporary and launching them from the main Application palette rather than from the Mathematica palettes window.
An Application is strongly enhanced with good documentation. The best kind of documentation is the regular Wolfram Research paclet documentation system. If you use this, your Application will blend smoothly with Mathematica. Otherwise, you are asking users to learn a subsidiary scheme within Mathematica.
The software for producing paclet documentation is the Wolfram Research Workbench. This comes with Premier Service and perhaps can be negotiated via other arrangements. The current version that works with Mathematica 10 is Workbench 3.0. (2015). Again, you may have to specially request this even if you have Premier Service.
Workbench has a number of other functionalities besides documentation including debugging. For the developers of Mathematica, who are writing a lot of C++ code interfacing with Wolfram Language Code, these facilities are probably quite useful. For regular uses of Mathematica they are less useful. Chances are you can write everything in the Wolfram Language, and it is easy enough to debug simply by putting Print statements in a problem routine. Therefore, I do all my development and debugging in regular Mathematica and use Workbench only for documentation. This is also consonant with the idea that the Application will grow out of notebooks used to study, explore or develop your subject matter. It keeps you closer to the math and away from “computer science” issues.
One advantage of confining Workbench to documentation is that you can keep all of your package files, style sheets, palettes and other notebooks out of the Workbench files. Well almost, as I’ll explain in a moment. One advantage of this is that you don’t have two sets of files, one deployed and one in Workbench. With two sets of files, if you make a change while using regular Mathematica you have to remember to update the corresponding Workbench file. Or you have to do all your work in Workbench. Otherwise, when you Deploy from Workbench you will overwrite your new files with the old ones.
I’m going to give only an outline of creating documentation and you will have to consult the Workbench documentation for details. Basically, once you get set up, it’s quite easy to add documentation pages.
To get started you create a new project with the New button in the menu. The project name might be MySubjectMatterProject. You also specify an Application name, MySubjectMatter, which is the top folder in the deployed Application. Workbench will create the folders and files that go with the documentation.
There are two places where you can set Preferences or Properties in Workbench. One is in the Menu Window/Preferences/Wolfram where you have to set the version of Mathematica you are using. The other is in the Menu Project/Properties/Wolfram where you can give paths to your deployed application files (Source, Palettes, Stylesheets). For example, Source would be MySubjectMatter. On the Preferences/Wolfram/Paclet Development click the button to Auto-Trust Documentation Palette. This palette will appear on the Mathematica Palettes menu once you’ve opened a documentation notebook. It’s called DocumentationTools there. It’s used to make links and format documentation pages.
One of the files that Workbench will create is the PacletInfo.m notebook. You can open this within Workbench by double clicking it. You will see the Overview tab. Fill that out. The name will again be the name of the top folder in your Application.
The object is then to create a Main Guide page in the Guides portion (and possible subsidiary Guide pages linked from the Main Guide page) and Symbol pages (also known as Function pages or Help pages) in the Symbols portion. I would suggest, at this stage not creating anything else. New pages are created by selecting the Symbols or Guides tab and using the Add Page button. The names of the Help pages should be the same as the names of the routines that are being documented. The Guide page names are up to you. If you are just beginning I would suggest creating just a couple Help pages and a Guide page that will link to them. The Help pages could just be in skeleton form and you can fill them out a little later, after you see that your Guide page links to them.
Open your new Guide page from the Documentation tab of PacletInfo.m. Give it a minute to format. (This, by the way, opens Mathematica proper.) Then from the Mathematica Palettes menu open DocumentationTools. It has tabs for Function (Help) pages labeled F and for Guide pages labeled G. Select the G tab. The name you gave to the Guide page will be near the top of the page. You can add some descriptive textual material beneath it. You can make a copy of one of the major sections (with a blue line at the top) so you have one to experiment with. Below the dotted line you can just start typing (on a new line) the names of the Function pages you created. Put a space-period-space between each of the page names. Then with the cursor anywhere in the cell, or with the cell bracket selected, use the DocumentationTools/Function Inline Listing button to make the links. If you then hover the mouse over the link it should display at the bottom of the window. It should look like “paclet:MySubject/ref/RoutineName”. If you click it, it should bring up the page. If this works you are basically set!
The next task is to fill out the Function (Help) pages. Bring up one by clicking its name on the Guide page, or from the PacletInfo/Documentation/Symbols tab. We have to fill in:
- The usage information in the tan usage box.
- The usage details under the usage box.
- The See Also section, which makes links to related routines.
- The Examples section.
To start, go to the Examples section and type in the Get command that loads your Application. (<< MySubjectMatter`). Load the package and evaluate the usage message for the routine being documented. Copy or move it near the top of the page and use it as a guide to fill in the Function page usage. Often the page usage box can be made shorter that the regular usage message because details can be moved to the Details entries just below the tan box.
Pick the F tab on the DocumentationTools palette. On the Function page, in the usage box, select the routine name along with the arguments and use the Template Input button to give them special formatting. The routine name and arguments should be on the first line of the usage box and the description on the following lines. You may also want to use Template Input on names of arguments or routines that appear in the description. You will often get a choice on whether the item should just be formatted or should also be a link. Most of the time you will not want a link. You may want to make a copy of the usage box so you can backtrack if you make too many mistakes.
Under the usage box you can just start a new cell and type text for the Details section of the Function page. You can put anything you want here, including how the routine fits into the larger application and maybe even references.
The next section to fill in is See Also. This is similar to the Inline links on the Guide page except now you will use the F tab on DocumentationTools and the Inline Listing Toggle. Start a new line under the See Also heading and type the names of related routines, separating them with space-period-space. Then use the Inline Listing Toggle. If you make a spelling error, say, you can toggle back and correct it. Click all the links to make sure they work. If you want to make a link to a Mathematica routine, say Plot, you can enter it as ref/Plot. These latter links will only be turned into working links when the Application goes through the Build process.
The Examples are the last section to complete. Here one can not only show basic usage but also illustrate a number of examples that fit into the subject matter. If the routine is commonly used for certain tasks then show them. You should have a number of existing examples from when the routine was first developed. Users who are beginners in the subject matter will look to the Function pages for broader information on the material so it’s a good place to include small lessons. You can also include examples that test the routines by illustrating the various usages and forms of data.
Once the pages are written you can do a Build and Deploy. Close all notebooks and the DocumentationTools palette. On the Workbench Application Tools panel select your project name. Minimize the PacletInfo.m panel so only the Console panel shows. Then click the Build button. Build is rather slow, often taking 10 to 15 minutes for a moderate size Application. Apparently it has to open every “raw” documentation notebook and reformat it as a finished documentation notebook. These will go into a “build” folder in your Workbench project files.
To Deploy the Application, Select the Project Wolfram Engine. Then click the Deploy Application Button. Select the $UserBaseDirector as the destination for the Application. This is where you can get into some trouble (as of mid 2015). If you have nothing other than documentation to Deploy, Workbench forces you into a Manual select mode. As far as I can tell the Manual select mode does not work properly and should be avoided like the plague. Therefore you need to export something in addition to the documentation. The easiest choice is the init.m file. This is usually simple and won’t change often. Nevertheless you should keep it up to date in Workbench and perhaps keep a duplicate initSave.m copy in your Application. With that, you can stay in the Auto select mode, Just click Next and Finish and the documentation will be installed or updated. This goes rather fast.
You Application folders will now look like:
$UserBaseDirectory/Applications/ MySubjectMatter Documentation FrontEnd/ Kernel/init.m StyleSheets/MySubjectMatter/ StylesheetPreview.nb EmptyStylesheet.nb MySubjectStyle1.nb Palettes/MySubjectMatter/ palette notebooks... SubjectMatterCode.m PublicFolders PrivateFolders
You now have a full-fledged deployed Application. It is an organized and flexible container for developing and preserving your material and communicating it to colleagues.