README
¶
README for the initializer
The initializer package supplies an Initializer struct which contains the data and supplies the methods necessary to initialize a Pipefish script, by creating and directing a parser and a compiler to operate on a VM. The Initializer returns a Compiler capable of dealing with runtime compilation of requests from the REPL: the Initializer can then be discarded together with its data.
There is one Initializer per module, as with compilers and parsers: an Initializer can spawn further Intitializers recursively to initialize modules and external services.
The initializer package consists of the following files:
-
api_deserializationis used to deserialize the APIs of external services. -
api_serializationis used to serialize the API of the service at compile time to supply to client services. -
externalscontains everything else the initializer needs to set up ways for the VM to use external services. -
function tabledefines theFunctionTabletype and methods for manipulating it. This is an intermediate step in the production of theFunctionTreeobjects that the compiler uses to perform multiple dispatch. -
getterssupplies some miscellaneous utility functions for getting and transforming data. -
gogen.gowhich generates Golang source files. -
gohandler.gowhich does housekeeping for the Go interop. -
initializer, the main file directing initialization. -
parsinghandles the early stages of intialization mostly concerned with setting up the parsers and parsing the code. -
pchunksdefines things satifying theparsedCodeinterface. These are structured bundles of parsed code and metadata and so on representing function/command declarations, constant/variable declarations, and type validation logic after the coe in them has been turned into ASTs -
tchunksdefines things satifying thetokenizedCodeinterface. These are structured bundles of tokens and containers containing tokens, etc, representing the various kinds of declaration as identified by Pipefish's headwords, functions/commands, type declarations, import declarations, etc.
Fields of note in the Initializer struct are its compiler (naturally); its parser (a shortcut to the parser of the compiler); Common, a bindle of data that all the initializers of all the modules need to share; and GoBucket, which is used to accumulate the miscellaneous data swept up during parsing that we need to generate Go source files.
Documentation
¶
Index ¶
- Constants
- Variables
- func AsBling(s string) parser.TypeNode
- func Equals(twp *parser.TypeWithParameters, twq *parser.TypeWithParameters) bool
- func ExtractFileName(s string) string
- func GetSourceCode(scriptFilepath string) (string, error)
- func MakeAstTypeFrom(s string) parser.TypeNode
- func MakeFilepath(scriptFilepath string) string
- func NewCommonInitializerBindle(store values.Map, services map[string]*compiler.Compiler) *commonInitializerBindle
- func StartCompiler(scriptFilepath, sourcecode string, hubServices map[string]*compiler.Compiler, ...) *compiler.Compiler
- func StartCompilerFromFilepath(filepath string, svs map[string]*compiler.Compiler, store values.Map) (*compiler.Compiler, error)
- func SummaryString(dec tokenizedCode) string
- func TweakNameAndPath(name, path, source string) (string, string)
- func Values(twa *parser.TypeWithArguments) []values.Value
- type ExternalCallToHubHandler
- type ExternalHttpCallHandler
- type Initializer
- func (iz *Initializer) Add(functionName string, f *parsedFunction) *parsedFunction
- func (iz *Initializer) AddInOrder(S []*parsedFunction, f *parsedFunction) ([]*parsedFunction, *parsedFunction)
- func (iz *Initializer) ChunkConstOrVarDeclaration(isConst, private bool, docString string) (tokenizedCode, bool)
- func (iz *Initializer) ChunkFunction(cmd, private bool, docString string) (*tokenizedFunctionDeclaration, bool)
- func (iz *Initializer) ChunkFunctionSignature() (*tokenizedFunctionDeclaration, bool)
- func (iz *Initializer) ChunkGolangDeclaration(private bool) (tokenizedCode, bool)
- func (iz *Initializer) ChunkImportOrExternalDeclaration(isExternal, private bool, docString string) (tokenizedCode, bool)
- func (iz *Initializer) ChunkTypeDeclaration(private bool, docString string) (tokenizedCode, bool)
- func (iz *Initializer) IsMandatoryImport(aT vm.AbstractTypeInfo) bool
- func (iz *Initializer) ParseEverythingFromFilePath(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, ...) (*compiler.Compiler, error)
- func (iz *Initializer) ParseEverythingFromSourcecode(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, ...) *compiler.Compiler
- func (iz *Initializer) SerializeApi() string
- func (iz *Initializer) SerializedAPIToDeclarations(serializedAPI string, xserve uint32) string
Constants ¶
const DUMMY = 4294967295
The maximum value of a `uint32`. Used as a dummy/sentinel value when `0` is not appropriate.
Variables ¶
var BUILTIN_FUNCTION_CONVERTER = map[string](func(t uint32, v any) any){ "bool": func(t uint32, v any) any { return v.(bool) }, "float": func(t uint32, v any) any { return v.(float64) }, "int": func(t uint32, v any) any { return v.(int) }, "rune": func(t uint32, v any) any { return v.(rune) }, "string": func(t uint32, v any) any { return v.(string) }, }
Maps for converting base types.
var BUILTIN_VALUE_CONVERTER = map[string]any{ "bool": (*bool)(nil), "float": (*float64)(nil), "int": (*int)(nil), "rune": (*rune)(nil), "string": (*string)(nil), }
var INTEROP_TOKEN = &token.Token{Source: "golang interop"}
var LINKING_TOKEN = &token.Token{Source: "Pipefish linker"}
Tokens to return when no token is available.
var PARSEABLE = []declarationType{cloneDeclaration, structDeclaration, constantDeclaration,
variableDeclaration, functionDeclaration, commandDeclaration}
Having declared the names of the various types and functions of the namespaces, we can now parse the chunks of actual code in the function bodies, `given` blocks, assignments, validation.
var StandardLibraries = dtypes.From[string]("crypto/aes", "crypto/bcrypt",
"crypto/rand", "crypto/rsa", "crypto/sha_256", "crypto/sha_512", "database/sql",
"encoding/csv", "encoding/base_32", "encoding/base_64", "encoding/json", "files",
"fmt", "html", "lists", "markdown", "math", "math/big", "math/cmplx", "math/rand",
"net/http", "net/mail", "net/smtp", "net/url", "os/exec", "path", "path/filepath",
"reflect", "regexp", "strings", "strconv", "terminal", "time", "unicode")
Functions ¶
func Equals ¶
func Equals(twp *parser.TypeWithParameters, twq *parser.TypeWithParameters) bool
func ExtractFileName ¶
func GetSourceCode ¶
func MakeAstTypeFrom ¶
func MakeFilepath ¶
func NewCommonInitializerBindle ¶
func NewCommonInitializerBindle(store values.Map, services map[string]*compiler.Compiler) *commonInitializerBindle
Initializes the `CommonInitializerBindle`.
func StartCompiler ¶
func StartCompilerFromFilepath ¶
func StartCompilerFromFilepath(filepath string, svs map[string]*compiler.Compiler, store values.Map) (*compiler.Compiler, error)
Initializes a compiler given the filepath.
func SummaryString ¶
func SummaryString(dec tokenizedCode) string
TODO --- we could fiddle with the api() method of the tokenized chunks to get most of this done byt that too.
func TweakNameAndPath ¶
Types ¶
type ExternalCallToHubHandler ¶
type ExternalCallToHubHandler struct {
Evaluator func(line string) values.Value
ProblemFn func() bool
SerializeApi func() string
}
func (ExternalCallToHubHandler) Evaluate ¶
func (ex ExternalCallToHubHandler) Evaluate(line string) values.Value
func (ExternalCallToHubHandler) GetAPI ¶
func (es ExternalCallToHubHandler) GetAPI() string
func (ExternalCallToHubHandler) Problem ¶
func (es ExternalCallToHubHandler) Problem() *err.Error
type ExternalHttpCallHandler ¶
type ExternalHttpCallHandler struct {
Host string
Service string
Username string
Password string
Deserializer func(valAsString string) values.Value
}
func (ExternalHttpCallHandler) Evaluate ¶
func (es ExternalHttpCallHandler) Evaluate(line string) values.Value
func (ExternalHttpCallHandler) GetAPI ¶
func (es ExternalHttpCallHandler) GetAPI() string
func (ExternalHttpCallHandler) Problem ¶
func (es ExternalHttpCallHandler) Problem() *err.Error
type Initializer ¶
type Initializer struct {
P *parser.Parser // The parser for the module being initialized.
Common *commonInitializerBindle // The information all the initializers have in Common.
// contains filtered or unexported fields
}
Definition of the Initializer type.
func NewInitializer ¶
func NewInitializer(common *commonInitializerBindle) *Initializer
Makes a new initializer.
func (*Initializer) Add ¶
func (iz *Initializer) Add(functionName string, f *parsedFunction) *parsedFunction
func (*Initializer) AddInOrder ¶
func (iz *Initializer) AddInOrder(S []*parsedFunction, f *parsedFunction) ([]*parsedFunction, *parsedFunction)
func (*Initializer) ChunkConstOrVarDeclaration ¶
func (iz *Initializer) ChunkConstOrVarDeclaration(isConst, private bool, docString string) (tokenizedCode, bool)
As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.
func (*Initializer) ChunkFunction ¶
func (iz *Initializer) ChunkFunction(cmd, private bool, docString string) (*tokenizedFunctionDeclaration, bool)
As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.
func (*Initializer) ChunkFunctionSignature ¶
func (iz *Initializer) ChunkFunctionSignature() (*tokenizedFunctionDeclaration, bool)
This wraps around chunkFunctionArguments and extracts the right name.
func (*Initializer) ChunkGolangDeclaration ¶
func (iz *Initializer) ChunkGolangDeclaration(private bool) (tokenizedCode, bool)
func (*Initializer) ChunkImportOrExternalDeclaration ¶
func (iz *Initializer) ChunkImportOrExternalDeclaration(isExternal, private bool, docString string) (tokenizedCode, bool)
As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.
func (*Initializer) ChunkTypeDeclaration ¶
func (iz *Initializer) ChunkTypeDeclaration(private bool, docString string) (tokenizedCode, bool)
As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.
func (*Initializer) IsMandatoryImport ¶
func (iz *Initializer) IsMandatoryImport(aT vm.AbstractTypeInfo) bool
func (*Initializer) ParseEverythingFromFilePath ¶
func (iz *Initializer) ParseEverythingFromFilePath(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, scriptFilepath, namespacePath string) (*compiler.Compiler, error)
Just exists to wrap around the next function.
func (*Initializer) ParseEverythingFromSourcecode ¶
func (iz *Initializer) ParseEverythingFromSourcecode(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, scriptFilepath, sourcecode, namespacePath string) *compiler.Compiler
This initializes the initializer's compiler (which initializes its parser), and extracts the source code from the given file, and then calls the `parseEverything“ method, below.
func (*Initializer) SerializeApi ¶
func (iz *Initializer) SerializeApi() string
For a description of the file format, see README-api-serialization.md
func (*Initializer) SerializedAPIToDeclarations ¶
func (iz *Initializer) SerializedAPIToDeclarations(serializedAPI string, xserve uint32) string
TODO --- malformed data would crash the deserializer with e.g. indexing errors. We need to make this a method of the initializer so it can Throw.
Source Files