KDP_book

Mathematica: Functional and procedural programming

At last, the procedure ContextsInFiles provides evaluation of the context ascribed to a file of the format {"m", "mx", "cdf", "nb"}. The procedure call ContextsInFiles[w] returns the single context ascribed to a file w of the above formats. At absence of a context the call returns $Failed. In addition, it must be kept in mind that the context in files of the specified format is sought relative to the key word "BeginPackage" that is typically used at beginning of a package. A return of list of format {"Context1", …, "Contextp"} is equivalent to existence in a m–file of a certain construction of format BeginPackage["context1'", {"context2'",

..., "contextp'"}] where {"context2'", ..., "contextp'"} determines uploadings of the appropriate files if their contexts aren't in the $Packages variable. The next fragment represents source code of the procedure with the most typical examples of its use.

In[47]:= ContextsInFiles[x_ /; FileExistsQ[x] && MemberQ[{"m", "mx", "cdf", "nb"}, FileExtension[x]]] :=

Module[{a = StringReplace[ReadFullFile[x], {" " –> "", "\n" –> "", "\t" –> ""}], b, b1, d = Flatten[{Range[65, 90], Range[96, 122]}]},

If[FileExtension[x] == "m", b = StringCases[a, Shortest["BeginPackage[\"" ~~ __ ~~ "`\"]"]]; If[b === {}, b = $Failed, b = Map[StringTake[#, {15, –3}] &, b][[1]];

b1 = StringCases[a, Shortest["BeginPackage[\"" ~~ __ ~~ "`\"}]"]]; If[b1 === {}, b1 = $Failed, If[! StringFreeQ[b1[[1]], {"{", "}"}], b1 = StringReplace[b1[[1]], {"{" –> "", "}" –> "", "\"" –> ""}]; b1 = StringSplit[StringTake[b1, {14, –2}], ","], b1 = $Failed]]];

b= DeleteDuplicates[Flatten[{b, b1}]];

SetAttributes[ContextQ, Listable];

b = Select[b, AllTrue[Flatten[ContextQ[{#}]], TrueQ] &];

If[ListQ[b] && Length[DeleteDuplicates[b]] == 1, b[[1]], b], If[FileExtension[x] == "mx", Quiet[Check[b = StringCases[a, Shortest["CONT" ~~ __ ~~ "ENDCONT"]][[1]]; b = Flatten[Map[ToCharacterCode, Characters[StringReplace[b, {"CONT" –> "", "ENDCONT" –> ""}]]]];

StringJoin[Map[FromCharacterCode[#] &, Select[b, MemberQ[d, #] &]]], $Failed]], Quiet[Check[a = StringCases[a, Shortest["BeginPackage\",

381

V.Z. Aladjev, M.L. Shishakov, V.A. Vaganov

\"[\",\"\\\"\\<" ~~ __ ~~ "`\\>\\\"\",\"]\"}]"]][[1]]; a = StringReplace[a, "BeginPackage" –> ""]; b = Flatten[Map[ToCharacterCode, Characters[a]]]; StringJoin[Map[FromCharacterCode[#] &, Select[b, MemberQ[d, #] &]]], $Failed]]]]]

In[48]:= ContextsInFiles["C:\\mathematica\\mathtoolbox.mx"]

Out[48]= "AladjevProcedures`"

In[49]:= ContextsInFiles[$InstallationDirectory <>\\SystemFiles\\ components\\ccodegenerator\\CCodeGenerator.m"]

Out[49]= {"CCodeGenerator`", "CompiledFunctionTools`", "CompiledFunctionTools`Opcodes`", "SymbolicC`", "CCompilerDriver`"}

In addition to the previous ContextsInFiles procedure the ContextsFromFiles procedure allows to extract contexts from files of any format that are located in the set directories or are given by own full or short names. The call ContextsFromFiles[] without arguments returns the sorted list of contexts from all files located in the catalog $InstallationDirectory and in all its subdirectories whereas the call ContextsFromFiles[x] returns the sorted list of contexts from all files located in a directory x and in all its sub-directories, at last the call ContextsFromFiles[x] on an existing x file returns the sorted list of contexts from the x file. The unsuccessful means call returns $Failed or is returned unevaluated. The next fragment represents source code of the procedure with the most typical examples of its application.

In[2226]:= ContextsFromFiles[x___String] :=

Module[{a = If[{x} == {}, FileNames["*", $InstallationDirectory,

Infinity], If[DirQ[x], FileNames["*", x, Infinity], If[FileExistsQ[x], {x}, $Failed]]], b}, If[a === $Failed, $Failed, b = Select[Map[Quiet[StringReplace[#, {": " -> "", " " -> "", "\"" -> ""}]] &,

Flatten[Map[StringCases[Quiet[Check[ReadFullFile[#], ""]], (Shortest[": " ~~ __ ~~ "` "] | Shortest["\"" ~~ __ ~~ "`\""]) ..] &, a]]], ContextQ[#] &]; Sort[DeleteDuplicates[b]]]]

In[2227]:= ContextsFromFiles[$InstallationDirectory <>

"\\AddOns\\Applications\\AuthorTools\\Kernel"]

Out[2227]= {"AuthorTools`", "AuthorTools`MakeBilateralCells`"}

Note, the above procedure operates on platforms Windows

XP Professional and Windows 7 Professional. The performance of the procedure is higher if it is applied to a mx–file.

382

Mathematica: Functional and procedural programming

The binary mx–files are optimized for fast loading. Wherein, they cannot be exchanged between different operating systems or versions of the Mathematica. Any mx–file contains version of Mathematica, the type of operating system in which it has been created and the context if it will exist. The procedure call ContextMathOsMx[x] returns the three–element list whose the first element determines the context if it exists (otherwise, $Failed is returned), the second element defines the list whose elements define version, releases of the Mathematica and the 3rd element defines operating system in which a mx–file x has been created.

In view of distinctions of the mx–files created on different platforms there is a natural expediency of creation of the tools testing a mx–file regarding the platform in which it was created in virtue of the DumpSave function. The TypeWinMx procedure is one of such tools. The procedure call TypeWinMx[x] in string format returns type of operating platform on which a mx–file x was created; correct result is returned for a case of the Windows platform, whereas on other platforms $Failed is returned. This is conditioned at absence of a possibility to carry out debugging on other platforms. The next fragment represents source code of the TypeWinMx procedure with examples of its application.

In[3339]:= TypeWinMx[x_ /; FileExistsQ[x] && FileExtension[x] == "mx"] := Module[{a, b, c, d},

If[StringFreeQ[$OperatingSystem, "Windows"], $Failed, a = StringJoin[Select[Characters[ReadString[x]], SymbolQ[#]||Quiet[IntegerQ[ToExpression[#]]]||# == "–" &]]; d = Map[FromCharacterCode, Range[2, 27]]; b = StringPosition[a, {"CONT", "ENDCONT"}];

If[b[[1]][[2]] == b[[2]][[2]],

c= StringCases1[a, {"Windows", "ENDCONT"}, "___"], b = StringPosition[a, {"Windows", "CONT"}];

c= StringTake[a, {b[[1]][[1]], b[[2]][[2]]}]];

c= StringReplace[c, Flatten[{GenRules[d, ""], "ENDCONT" –> "", "CONT" –> ""}]]; If[ListQ[c], c[[1]], c]]]

In[3340]:= TypeWinMx["c:\\mathematica\\mathToolBox.mx"]

Out[3340]= "Windows–x86–64"

383

V.Z. Aladjev, M.L. Shishakov, V.A. Vaganov

Along with the problem of determining the context in files of format {m, mx, nb}, the problem of determining files of this type containing the given context is of particular interest. This task is solved by the procedure whose call FilesWithContext[x, y] returns the list containing full paths to files from a directory y and all its sub-directories of the format {m,mx,nb} that contain a context x. Argument y is optional and in case of its absence instead of it the directory Directory[] is used. The procedure call with incorrect directory y returns $Failed, whereas the absence in the directory y files with demanded context x returns empty list. Note that the procedure uses our procedure, whose call StandPath[w] returns the result of standardizing the path to a file or directory w, which ensures that the FilesWithContext procedure runs correctly on directories and files whose names contain space symbol. In the following fragment the source code of the procedure and examples of its application is represented.

In[42]:= FilesWithContext[x_ /; ContextQ[x], y___] := Module[{a, b, c, d, g, h},

If[{y} == {}, a = Directory[], If[! DirQ[y], Return[$Failed], a = y]]; b = Run["Dir " <> a <> "\\*.mx /B/S > " <> Directory[] <> "\\#"]; a = StandPath[a]; If[b == 0, c = ReadList["#", String], c = {}];

b = Run["Dir " <> a <> "\\*.m /B/S > " <> Directory[] <> "\\#"]; If[b == 0, d = ReadList["#", String], d = {}]; b = Run["Dir " <> a <> "\\*.nb /B/S > " <> Directory[] <> "\\#"];

If[b == 0, g = ReadList["#", String], g = {}]; DeleteFile["#"]; If[Set[h, Join[c, d]] == {}, {}, c = Map[StringReplace[#, " . " –> "."] &, h]];

Map[If[ContextFromFile[#] === x, #, Nothing] &, c]] In[43]:= FilesWithContext["AVZ`", Directory[]]

Out[43]= {"C:\\Users\\Aladjev\\Documents\\$$avzagn.mx", "C:\\Users\\Aladjev\\Documents\\Agn\\$$avzagn.m"}

In[44]:= FilesWithContext["RANS`"]

Out[44]= {"C:\\Users\\Aladjev\\Documents\\$avzagn.mx", "C:\\Users\\Aladjev\\Documents\\Agn\\$avzagn.m"}

This procedure naturally extends to files of certain other types. A quite certain interest is the question of occurrence in

384

Mathematica: Functional and procedural programming

the definition of the user tools of the tools calls other than builtin tools. The FullContent procedure solves this problem. Calling

FullContent[x] procedure returns the sorted list containing all names in string format of the user tools whose calls are in definition of a tool x. Whereas the call FullContent[x, y] with the 2nd optional y argument

– an indefinite symbol – additionally through it returns the nested list of the above tools with contexts ascribed to them; the list is gathered according to the contexts. In addition, a context is the first in the list of the tools names with this context. Indeed, the name x is included in the returned list too. The following fragment represents the source code of the FullContent procedure with examples of its application.

In[42]:= FullContent[x_, y___] := Module[{a, b, c, n, m},

If[MemberQ[{FullContent, "FullContent"}, x], c = FullContent[ProcQ], a = Save["###", x]; a = StringReplace[ReadString["###"], {"\n" –> "", "\r" –> ""}];

b = StringCases[a, Shortest[" /: " ~~ __ ~~ "::usage ="]]; DeleteFile["#"]; c = Map[{n = StringReplace[#, "::usage =" –> ""]; m = Flatten[StringPosition[n, " /: "]]; StringTake[n, {m[[–1]] + 1, –1}]} &, b];

c = Sort[DeleteDuplicates[Map[StringTrim[#, "\\"] &, Flatten[c]]]]]; If[{y} != {} && ! HowAct[y], y = Map[{#, Context[#]} &, c]; y = Gather[y, #1[[2]] == #2[[2]] &]; y = Sort[Map[Sort[DeleteDuplicates[Flatten[#]]] &, y], If[ContextQ[#1], #1, #2] &]; c, c]]

In[43]:= FullContent[ProcQ, gs]

Out[43]= {"Attributes1", "BlockFuncModQ", "ClearAllAttributes", "Contexts1", "Definition2", "HeadPF", "HowAct", "ListStrToStr", "Map3", "Mapp", "MinusList", "ProcQ", "ReduceLists", "ProtectedQ", "PureDefinition", "StrDelEnds", "SuffPref", "SymbolQ", "SysFuncQ", "SystemQ", "ToString1", "UnevaluatedQ"}

In[44]:= gs

Out[44]= {{"AladjevProcedures`", "Attributes1", …, "Map3", …, "SymbolQ", "SysFuncQ", "SystemQ", "ToString1", "UnevaluatedQ"}}

In[45]:= {FullContent[LoadMxQ, vg], vg}

Out[45]= {{"LoadMxQ", "MxOnOpSys", "StrDelEnds", "StringCases2",

"SuffPref", "ToString1"}, {{"AladjevProcedures`", "LoadMxQ", "MxOnOpSys", "StrDelEnds", "StringCases2", "SuffPref", "ToString1"}}

The m–format is typical package format. As a rule it is used for the distribution of both system and user packages. The next

385

V.Z. Aladjev, M.L. Shishakov, V.A. Vaganov

rather simple procedure allows to retrieving the contents of such user files without loading them into the current session. Calling the ContentsM[x] procedure returns the list of names in string format that compose the user m–format file x without loading it into the current session. The following fragment represents the source code of the ContentsM procedure and examples of its use.

In[1940]:= ContentsM[x_ /; FileExistsQ[x] &&

FileExtension[x] == "m"] := Module[{a = ReadFullFile[x], j}, j = StringCases[a, Shortest["(*Begin[\"`" ~~ __ ~~ "`\"]*)"]]; j = Map[StringReplace[#, {"(*Begin[\"`"-> "", "`\"]*)" -> ""}] &, j];

Sort[Map[If[SymbolQ[#], #, Nothing] &, j]]]

In[1941]:= ContentsM["C:\\Math\\mathtoolbox.m"]

Out[1941]= {"AcNb", "ActBFM", "ActBFMuserQ", "ActCsProcFunc", "ActivateMeansFromCdfNb", "ActRemObj", "ActUcontexts",…,"$UserContexts2", "$Version1", "$Version2"}

In[1942]:= Length[%]

Out[1942]= 1421

It is known, Mathematica language code and expressions can be stored in a variety of formats. In particular, {.wl, .m} file formats are used to store packages. As a rule they are used for the distribution of system and user packages. A rather simple procedure allows to retrieving the contents of the user wl–files without loading them into the current Mathematica session. Calling the ContentsWl[x] procedure returns the nested list of names in string format of blocks, functions and modules that compose the user wl–format file x without loading it into the current session. The elements of the returned list – 2–elements sub–lists whose the first element is a context whereas the other elements are names corresponding it. The following fragment represents the source code of the ContentsWl procedure with some typical examples of its application.

In[78]:= ContentsWl[x_ /; FileExistsQ[x] && FileExtension[x] == "wl"] := Module[{s, a, b, c, g, v}, s = ReadString[x]; a = StringSplit[s, {"(* ::Input:: *)", "(* ::Package:: *)"}];

b = StringReplace[a, {"(*" –> "", "*)" –> "", "\r" –> "", "\n" –> ""}]; c = Map[If[# != "" && SyntaxQ[#], #, Nothing] &, b];

386

Mathematica: Functional and procedural programming

g = Map[{g = Flatten[StringPosition[#, ":=", 1]]; If[g == {}, Nothing, If[HeadingQ4[v = StringTake[#, {1, g[[1]] – 1}]],

v, Nothing]]} &, c]; g = DeleteDuplicates[Flatten[g]]; g = Map[HeadName1[#] &, g]; g = Map[{#, Context[#]} &, g]; g = Gather[g, #1[[2]] == #2[[2]] &]; g = Map[Flatten, g]; g = Map[Sort[DeleteDuplicates[#]] &, g]; Map[Sort[#, ContextQ[#] &] &, g]]

In[79]:= ContentsWl["C:\\Mathematica\\Exp79.wl"]

Out[79]= {{"AladjevProcedures`", "ToUnique", "Replace7", "RandSortList", "PrevUnique", "PartSortList", "Agn"}, {"Global`", "ReplaceInSitu", "P", "Mn", "MM", "G"}}

As an auxiliary tool, the procedure uses a procedure whose call HeadingQ4[x] returns True if x is a header in string format of a block, function, or module, and False otherwise. The source code of the HeadingQ4 procedure along with an example of its application are represented below.

In[84]:= HeadingQ4[x_ /; StringQ[x]] := Module[{i, j}, Quiet[ToExpression[j= ToString[i] <> x; j<> ":=Module[{j},j]"]];

BlockFuncModQ[HeadName1[j]]]

In[85]:= HeadingQ4["DefFromMfile[x_ /; SymbolQ[x],

y_ /; FileExistsQ[y] && FileExtension[y] == \"wl\"]"]

Out[85]= True

Unlike the previous procedure, calling the next procedure DefFromMfile[x, y] returns the definition in the string format of the x symbol contained in the user m–file y, if it is not present, $Failed is returned.

In[47]:= DefFromMfile[x_ /; SymbolQ[x],

y_ /; FileExistsQ[y] && FileExtension[y] == "m"] := Module[{a, b, c, d, j, g = ""}, a = ReadFullFile[y]; b = "(*Begin[\"`" <> ToString[x] <> "`\"]*)";

If[StringFreeQ[a, b], $Failed, c = StringPosition[a, b]; c = Flatten[c][[2]]; Do[For[j = c + 1, j <= Infinity, j++, If[SyntaxQ[d = StringTake[a, {c + 1, j}]],

g = g <> StringTake[d, {3, –3}]; Break[], Continue[]]];

If[SyntaxQ[g], Return[g], c = j; Continue[]], Infinity]]]

In[48]:= DefFromMfile[StrStr, "C:\\Math\\MathToolBox.m"]

387

V.Z. Aladjev, M.L. Shishakov, V.A. Vaganov

Out[48]= "StrStr[x_] := If[StringQ[x], \"\\\"\"<>x<>\"\\\"\",

ToString[x]]"

Unlike the ContentsM procedure, calling the following procedure ContentsNb[x] returns the nested list of names in string format that compose the user nb–format file x without loading it into the current session. In addition, the first element of the returned list – the list of the system names, the second element – the list whose sub–lists contains contexts with names of blocks, functions or modules corresponding them, and the third element – the list of other names in string format with preceding them contexts. The fragment below represents the source code of the ContentsNb procedure with typical examples of its application.

In[54]:= ContentsNb[x_ /; FileExistsQ[x] &&

FileExtension[x] == "nb"] := Module[{a = Quiet[ToString1[Get[x]]], b, c, d = {{}, {}, {}}}, b = StringCases[a, Shortest["RowBox[{" ~~ __ ~~ ", "]];

b = Map[StringReplace[#, {"RowBox[{"–>"", " "–>"", ", "–>""}] &, b]; c = Sort[Select[b, SymbolQ[#] &]]; c = DeleteDuplicates[Map[ToExpression, c]];

Map[If[SystemQ[#], AppendTo[d[[1]], #], If[BlockFuncModQ[#], AppendTo[d[[2]], #], AppendTo[d[[3]], #]]] &, c]; {d[[1]], Map[Sort[DeleteDuplicates[Flatten[Gather[#,

#1[[2]] == #2[[2]] &]]], ContextQ[#] &] &, {d[[2]], d[[3]]}]}]

In[55]:= ContentsNb["C:\\Mathematica\\Exp78.nb"]

Out[55]= {{"AppendTo", "Break", "Continue", "DeleteDuplicates", "Do", "FileExistsQ", "FileExtension", "Flatten", "For", "Get", "If", "Map", "Module", "Return", "Select", "Shortest", "Sort", "StringCases", "StringFreeQ", "StringPosition", "StringReplace", "StringTake", "SyntaxQ", "ToExpression", "ToString"}, {{"AladjevProcedures`", "ToString1", "SystemQ", "SymbolQ", "ReadFullFile", "ProcQ", "NbToString", "DefFromMfile", "ContentsNb", "BlockFuncModQ"}, {"AladjevProcedures`", "Global`", "j", "s", "g", "d", "c", "b", "a"}}}

Similarly to the procedure ContentsNb, the procedure call ContentsMx1[x] returns the nested list of names in string format that compose the user mx–format file x without loading it into

388

Mathematica: Functional and procedural programming

the current session. In addition, the first element of the returned list – the list of the system names, the second element – the list that contains names in string format of functions, modules and blocks, and the third element – the list of other names in string format. It is assumed that the mx–file contains the context and definitions of blocks, functions, and/or modules along with their usage.

In[10]:= ContentsMx1[x_/; FileExistsQ[x] && FileExtension[x] == "mx"] := Module[{a = BinaryReadList[x], b, c, d = {{}, {}, {}}}, b = Flatten[{34, 37, Map[Range5[#] &, {0 ;; 31, 40 ;; 47, 58 ;; 64, 92 ;; 95, 123 ;; 126}]}]; c = Map[If[MemberQ[b, #] || # >= 127, Nothing, #] &, a];

c = StringJoin[Map[If[# == 32, "|", FromCharacterCode[#]] &, c]]; c = StringCases[c, Shortest["|" ~~ __ ~~ "["]]; c = Map[StringTake[#, {Flatten[StringPosition[#, "|"]][[-1]]+1, -2}] &, c];

c = DeleteDuplicates[Sort[Select[c, SymbolQ[#] && StringFreeQ[#, {"`", "$"}] &]]]; c = Select[c, Context[#] != "Global`" &];

Map[If[SystemQ[#], AppendTo[d[[1]], #], If[BlockFuncModQ[#], AppendTo[d[[2]], #], AppendTo[d[[3]], #]]] &, c]; d]

In[11]:= ContentsMx1["C:\\Mathem\\mathtoolbox.mx"]

Out[11]= {{"Abs", "And", "Append",…,"Unique", "Xnor", "Xor"}, {"AcNb", "ActBFM", "ActBFMuserQ",…,"Xnor1", "Xor1", "XOR1"},

{"a", "f", "g", "x", "y", "z"}}

At last, the procedure call ContentsTxt1[x] returns the list of names in string format that composes the user txt–format file x with a package without loading it into the current session.

In[77]:= ContentsTxt1[x_/; FileExistsQ[x] && FileExtension[x] == "txt"] := Module[{a = ReadFullFile[x]}, a = StringCases[a, Shortest["Begin[" ~~ __ ~~ "]"]];

a = Map[StringReplace[#, {"Begin[" -> "", "`" -> "", "]" -> ""}] &, a];

Select[Map[ToExpression, Sort[Select[a, SyntaxQ[#] &]]], Quiet[SyntaxQ[#]] &]]

In[78]:= Length[ContentsTxt1["C:\\Math\\mathtoolbox.txt"]]

Out[78]= 1433

389

V.Z. Aladjev, M.L. Shishakov, V.A. Vaganov

The mx–format is serialized package format. As a rule it is used for the distribution of both system and user packages. The binary file format is the optimized for fast loading. Meantime, as noted above, the mx–format files can't be exchanged between operating platforms which differ from the predefined variable $SystemWordLength which gives the number of bits in machine words on the computer system where Mathematica is running. Furthermore, the mx–format files created by newer versions of Mathematica may not be usable by older versions. This raises the issue of testing the ability to load an arbitrary file of the mx- format into the current session. The following procedure solves this problem. The fragment below represents the source code of the LoadMxQ procedure with examples of its application.

In[7]:= LoadMxQ[x_ /; FileExistsQ[x] && FileExtension[x] == "mx"] :=

Module[{a, b, c, MxOnOpSys},

MxOnOpSys[y_/; FileExistsQ[y] && FileExtension[y] == "mx"] := Module[{a = ReadString[y], b = "Get.*)", c = "CONT", d}, d = StringCases2[a, {b, c}][[1]];

d = StringReplace[d, {b –> "", c –> "", "\n" –> "", "END" –> ""}]; d = StringJoin[Map[If[MemberQ[Range[32, 127], ToCharacterCode[#][[1]]], #, Nothing] &, Characters[d]]]]; g[t_] := t; DumpSave["###.mx", g]; If[MxOnOpSys[x] == MxOnOpSys["###.mx"],

DeleteFile["###.mx"]; True, DeleteFile["###.mx"]; False]]

In[8]:= LoadMxQ["C:\\Mathematica\\MathToolBox.mx"]

Out[8]= True

In[9]:= LoadMxQ["dump.mx"]

Out[9]= True

Calling procedure LoadMxQ[x] returns True if a mx–type file x was created on the current operating platform and can be loaded into the current Mathematica session by the Get[x] call, and False otherwise. In our works [8-16] a number of means for various processing of mx–files with interesting examples using in programming of various applications is presented. However, these tools have been debugged on Windows XP and Windows 7 platforms, in other cases requiring sometimes re–debugging.

390