Apache::lonnet.pm
This file is an interface to the lonc processes of the LON-CAPA network as well as set of elaborated functions for handling information necessary for navigating through a given cluster of LON-CAPA machines within a domain. There are over 40 specialized functions in this module which handle the reading and transmission of metadata, user information (ids, names, environments, roles, logs), file information (storage, reading, directories, extensions, replication, embedded styles and descriptors), educational resources (course descriptions, section names and numbers), url hashing (to assign roles on a url basis), and translating abbreviated symbols to and from more descriptive phrases or explanations.
This is part of the LearningOnline Network with CAPA project described at http://www.lon-capa.org.
These are largely undocumented, so if you decipher one please note it here.
Contains the time this process was started and this servers host id.
Counts the number of times a message log flush has been attempted (regardless of success) by this process. Used as part of the filename when messages are delayed.
Apache::lonnet - Subroutines to ask questions about things in the network.
Invoked by other LON-CAPA modules, when they need to talk to or about objects in the network.
&Apache::lonnet::SUBROUTINENAME(ARGUMENTS);
Common parameters:
$uname : an internal username (if $cname expecting a course Id specifically)
$udom : a domain (if $cdom expecting a course's domain specifically)
$symb : a resource instance identifier
$namespace : the name of a .db file that contains the data needed or being set.
lonnet provides subroutines which interact with the lonc/lond (TCP) network layer of LON-CAPA. They can be used to ask about classes, users, and resources.
For many of these objects you can also use this to store data about them or modify them in various ways.
To identify a specific instance of a resource, LON-CAPA uses symbols or ``symbs''. These identifiers are built from the URL of the map, the resource number of the resource in the map, and the URL of the resource itself. The latter is somewhat redundant, but might help if maps change.
An example is
msu/korte/parts/part1.sequence___19___msu/korte/tests/part12.problem
The respective map entry is
<resource id="19" src="/res/msu/korte/tests/part12.problem" title="Problem 2"> </resource>
Symbs are used by the random number generator, as well as to store and restore data specific to a certain instance of for example a problem.
Three of the most important functions
in lonnet.pm
are &Apache::lonnet::cstore()
,
&Apache::lonnet:restore()
, and &Apache::lonnet::store()
, which
is is the non-critical message twin of cstore. These functions are for
handlers to store a perl hash to a user's permanent data space in an
easy manner, and to retrieve it again on another call. It is expected
that a handler would use this once at the beginning to retrieve data,
and then again once at the end to send only the new data back.
The data is stored in the user's data directory on the user's homeserver under the ID of the course.
The hash that is returned by restore will have all of the previous value for all of the elements of the hash.
Example:
#creating a hash my %hash; $hash{'foo'}='bar';
#storing it &Apache::lonnet::cstore(\%hash);
#changing a value $hash{'foo'}='notbar';
#adding a new value $hash{'bar'}='foo'; &Apache::lonnet::cstore(\%hash);
#retrieving the hash my %history=&Apache::lonnet::restore();
#print the hash foreach my $key (sort(keys(%history))) { print("\%history{$key} = $history{$key}"); }
Will print out:
%history{1:foo} = bar %history{1:keys} = foo:timestamp %history{1:timestamp} = 990455579 %history{2:bar} = foo %history{2:foo} = notbar %history{2:keys} = foo:bar:timestamp %history{2:timestamp} = 990455580 %history{bar} = foo %history{foo} = notbar %history{timestamp} = 990455580 %history{version} = 2
Note that the special hash entries keys
, version
and
timestamp
were added to the hash. version
will be equal to the
total number of versions of the data that have been stored. The
timestamp
attribute will be the UNIX time the hash was
stored. keys
is available in every historical section to list which
keys were added or changed at a specific historical revision of a
hash.
Warning: do not store the hash that restore returns directly. This will cause a mess since it will restore the historical keys as if the were new keys. I.E. 1:foo will become 1:1:foo etc.
Calling convention:
my %record=&Apache::lonnet::restore($symb,$courseid,$domain,$uname,$home); &Apache::lonnet::cstore(\%newrecord,$symb,$courseid,$domain,$uname,$home);
For more detailed information, see lonnet specific documentation.
get_env_multiple($name)
gets $name from the %env hash, it seemlessly handles the cases where multiple values may be defined and end up as an array ref.
returns an array of values
$upass
should be the users password.
$checkdefauth is optional (value is 1 if a check should be made to
authenticate user using default authentication method, and allow
account creation if username does not have account in the domain).
$clientcancheckhost is optional (value is 1 if checking whether the
server can host will occur on the client side in lonauth.pm).
allowed($priv,$uri,$symb,$role)
: check for a user privilege; returns codes for allowed actions
F: full access
U,I,K: authentication modes (cxx only)
'': forbidden
1: user needs to choose course
2: browse allowed
A: passphrase authentication needed
definerole($rolename,$sysrole,$domrole,$courole)
: define role; define a custom
role rolename set privileges in format of lonTabs/roles.tab for system, domain,
and course level
plaintext($short,$type,$cid,$forcedefault)
: return value in %prp hash
(rolesplain.tab); plain text explanation of a user role term.
$type is Course (default) or Community.
If $forcedefault evaluates to true, text returned will be default
text for $type. Otherwise, if this is a course, the text returned
will be a custom name for the role (if defined in the course's
environment). If no custom name is defined the default is returned.
=item *
get_my_roles($uname,$udom,$context,$types,$roles,$roledoms,$withsec)
:
All arguments are optional. Returns a hash of a roles, either for
co-author/assistant author roles for a user's Construction Space
(default), or if $context is 'userroles', roles for the user himself,
In the hash, keys are set to colon-separated $uname,$udom,$role, and
(optionally) if $withsec is true, a fourth colon-separated item - $section.
For each key, value is set to colon-separated start and end times for
the role. If no username and domain are specified, will default to
current user/domain. Types, roles, and roledoms are references to arrays
of role statuses (active, future or previous), roles
(e.g., cc,in, st etc.) and domains of the roles which can be used
to restrict the list of roles reported. If no array ref is
provided for types, will default to return only active roles.
assignrole($udom,$uname,$url,$role,$end,$start,$deleteflag,$selfenroll,$context)
: assign role; give a role to a
user for the level given by URL. Optional start and end dates (leave empty
string or zero for ``no date'')
changepass($uname,$udom,$currentpass,$newpass,$server)
: attempts to
change a users, password, possible return values are: ok,
pwchange_failure, non_authorized, auth_mode_error, unknown_user,
refused
modifyuserauth($udom,$uname,$umode,$upass)
: modify user authentication
modifyuser($udom,$uname,$uid,$umode,$upass,$first,$middle,$last, $gene, $forceid,$desiredhome,$email,$inststatus,$candelete) :
will update user information (firstname,middlename,lastname,generation,
permanentemail), and if forceid is true, student/employee ID also.
A user's institutional affiliation(s)
can also be updated.
User information fields will not be overwritten with empty entries
unless the field is included in the $candelete array reference.
This array is included when a single user is modified via ``Manage Users'',
or when Autoupdate.pl is run by cron in a domain.
modifystudent
modify a student's enrollment and identification information. The course id is resolved based on the current users environment. This means the envoking user must be a course coordinator or otherwise associated with a course.
This call is essentially a wrapper for lonnet::modifyuser and lonnet::modify_student_enrollment
Inputs:
modify_student_enrollment
Change a students enrollment status in a class. The environment variable 'role.request.course' must be defined for this function to proceed.
Inputs:
assigncustomrole($udom,$uname,$url,$rdom,$rnam,$rolename,$end,$start)
: assign
custom role; give a custom role to a user for the level given by URL. Specify
name and domain of role author, and role name
revokerole($udom,$uname,$url,$role)
: revoke a role for url
revokecustomrole($udom,$uname,$url,$role)
: revoke a custom role
coursedescription($courseid)
: returns a hash of information about the
specified course id, including all environment settings for the
course, the description of the course will be in the hash under the
key 'description'
resdata($name,$domain,$type,@which)
: request for current parameter
setting for a specific $type, where $type is either 'course' or 'user',
@what should be a list of parameters to ask about. This routine caches
answers for 5 minutes.
get_courseresdata($courseid, $domain) : dump the entire course resource data base, returning a hash that is keyed by the resource name and has values that are the resource value. I believe that the timestamps and versions are also returned.
writecoursepref($courseid,%prefs)
: write preferences (environment
database) for a course
createcourse($udom,$description,$url,$course_server,$nonstandard,$inst_code,$course_owner,$crstype,$cnum)
: make course
generate_coursenum($udom,$crstype)
: get a unique (unused) course number in domain $udom for course type $crstype (Course or Community).
subscribe($fname)
: subscribe to a resource, returns URL if possible (probably should use repcopy instead)
repcopy($filename)
: subscribes to the requested file, and attempts to
replicate from the owning library server, Might return
'unavailable', 'not_found', 'forbidden', 'ok', or
'bad_request', also attempts to grab the metadata for the
resource. Expects the local filesystem pathname
(/home/httpd/html/res/....)
EXT($varname,$symb,$udom,$uname)
: evaluates and returns the value of
a vairety of different possible values, $varname should be a request
string, and the other parameters can be used to specify who and what
one is asking about.
Possible values for $varname are environment.lastname (or other item from the envirnment hash), user.name (or someother aspect about the user), resource.0.maxtries (or some other part and parameter of a resource)
directcondval($number)
: get current value of a condition; reads from a state
string
condval($condidx)
: value of condition index based on state
metadata($uri,$what,$liburi,$prefix,$depthcount)
: request a
resource's metadata, $what should be either a specific key, or either
'keys' (to get a list of possible keys) or 'packages' to get a list of
packages that this resource currently uses, the last 3 arguments are only used internally for recursive metadata.
this function automatically caches all requests
metadata_query($query,$custom,$customshow)
: make a metadata query against the
network of library servers; returns file handle of where SQL and regex results
will be stored for query
symbread($filename)
: return symbolic list entry (filename argument optional);
returns the data handle
symbverify($symb,$thisfn)
: verifies that $symb actually exists and is
a possible symb for the URL in $thisfn, and if is an encryypted
resource that the user accessed using /enc/ returns a 1 on success, 0
on failure, user must be in a course, as it assumes the existance of
the course initial hash, and uses $env('request.course.id'}
symbclean($symb)
: removes versions numbers from a symb, returns the
cleaned symb
is_on_map($uri)
: checks if the $uri is somewhere on the current
course map, user must be in a course for it to work.
numval($salt)
: return random seed value (addend for rndseed)
rndseed($symb,$courseid,$udom,$uname)
: create a random sum; returns
a random seed, all arguments are optional, if they aren't sent it uses the
environment to derive them. Note: if symb isn't sent and it can't get one
from &symbread it will use the current time as its return value
ireceipt($funame,$fudom,$fucourseid,$fusymb)
: return unique,
unfakeable, receipt
receipt()
: API to ireceipt working off of env values; given out to users
countacc($url)
: count the number of accesses to a given URL
checkout($symb,$tuname,$tudom,$tcrsid)
: creates a record of a user having looked at an item, most likely printed out or otherwise using a resource
checkin($token)
: updates that a resource has beeen returned (a hard copy version for instance) and returns the data that $token was Checkout with ($symb, $tuname, $tudom, and $tcrsid)
expirespread($uname,$udom,$stype,$usymb)
: set expire date for spreadsheet
devalidate($symb)
: devalidate temporary spreadsheet calculations,
forcing spreadsheet to reevaluate the resource scores next time.
store($storehash,$symb,$namespace,$udom,$uname)
: stores hash permanently
for this url; hashref needs to be given and should be a \%hashname; the
remaining args aren't required and if they aren't passed or are '' they will
be derived from the env
cstore($storehash,$symb,$namespace,$udom,$uname)
: same as store but
uses critical subroutine
restore($symb,$namespace,$udom,$uname)
: returns hash for this symb;
all args are optional
dumpstore($namespace,$udom,$uname,$regexp,$range)
:
dumps the complete (or key matching regexp) namespace into a hash
($udom, $uname, $regexp, $range are optional) for a namespace that is
normally &store()ed into
$range should be either an integer '100' (give me the first 100 matching records) or be two integers sperated by a - with no spaces '30-50' (give me the 30th through the 50th matching records)
putstore($namespace,$symb,$version,$storehash,$udomain,$uname)
:
replaces a &store() version of data with a replacement set of data
for a particular resource in a namespace passed in the $storehash hash
reference
tmpstore($storehash,$symb,$namespace,$udom,$uname)
: storage that
works very similar to store/cstore, but all data is stored in a
temporary location and can be reset using tmpreset, $storehash should
be a hash reference, returns nothing on success
tmprestore($symb,$namespace,$udom,$uname)
: storage that works very
similar to restore, but all data is stored in a temporary location and
can be reset using tmpreset. Returns a hash of values on success,
error string otherwise.
tmpreset($symb,$namespace,$udom,$uname)
: temporary storage reset,
deltes all keys for $symb form the temporary storage hash.
get($namespace,$storearr,$udom,$uname)
: returns hash with keys from array
reference filled in from namesp ($udom and $uname are optional)
del($namespace,$storearr,$udom,$uname)
: deletes keys out of array from
namesp ($udom and $uname are optional)
dump($namespace,$udom,$uname,$regexp,$range)
:
dumps the complete (or key matching regexp) namespace into a hash
($udom, $uname, $regexp, $range are optional)
$range should be either an integer '100' (give me the first 100 matching records) or be two integers sperated by a - with no spaces '30-50' (give me the 30th through the 50th matching records) =item *
inc($namespace,$store,$udom,$uname)
: increments $store in $namespace.
$store can be a scalar, an array reference, or if the amount to be
incremented is > 1, a hash reference.
($udom and $uname are optional)
put($namespace,$storehash,$udom,$uname)
: stores hash in namesp
($udom and $uname are optional)
cput($namespace,$storehash,$udom,$uname)
: critical put
($udom and $uname are optional)
newput($namespace,$storehash,$udom,$uname)
:
Attempts to store the items in the $storehash, but only if they don't currently exist, if this succeeds you can be certain that you have successfully created a new key value pair in the $namespace db.
Args: $namespace: name of database to store values to $storehash: hashref to store to the db $udom: (optional) domain of user containing the db $uname: (optional) name of user caontaining the db
Returns: 'ok' -> succeeded in storing all keys of $storehash 'key_exists: <key>' -> failed to anything out of $storehash, as at least <key> already existed in the db (other requested keys may also already exist) 'error: <msg>' -> unable to tie the DB or other error occurred 'con_lost' -> unable to contact request server 'refused' -> action was not allowed by remote machine
eget($namespace,$storearr,$udom,$uname)
: returns hash with keys from array
reference filled in from namesp (encrypts the return communication)
($udom and $uname are optional)
log($udom,$name,$home,$message)
: write to permanent log for user; use
critical subroutine
get_dom($namespace,$storearr,$udom,$uhome)
: returns hash with keys from
array reference filled in from namespace found in domain level on either
specified domain server ($uhome) or primary domain server ($udom and $uhome are optional).
put_dom($namespace,$storehash,$udom,$uhome)
: stores hash in namespace at
domain level either on specified domain server ($uhome) or primary domain
server ($udom and $uhome are optional)
get_domain_defaults($target_domain)
: returns hash with defaults for
authentication and language in the domain. Keys are: auth_def, auth_arg_def,
lang_def; corresponsing values are authentication type (internal, krb4, krb5,
or localauth), initial password or a kerberos realm, language (e.g., en-us).
Values are retrieved from cache (if current), or from domain's configuration.db
(if available), or lastly from values in lonTabs/dns_domain,tab,
or lonTabs/domain.tab.
%domdefaults = &get_auth_defaults($target_domain);
dirlist($uri)
: return directory list based on URI
spareserver()
: find server with least workload from spare.tab
host_from_dns($dns)
: Returns the loncapa hostname corresponding to a DNS name or undef
if there is no corresponding loncapa host.
ssi($url,%hash)
: server side include, does a complete request cycle on url to
localhost, posts hash
hash2str(%hash)
: convert a hash into a string complete with escaping and '='
and '&' separators, supports elements that are arrayrefs and hashrefs
hashref2str($hashref)
: convert a hashref into a string complete with
escaping and '=' and '&' separators, supports elements that are
arrayrefs and hashrefs
arrayref2str($arrayref)
: convert an arrayref into a string complete
with escaping and '&' separators, supports elements that are arrayrefs
and hashrefs
str2hash($string)
: convert string to hash using unescaping and
splitting on '=' and '&', supports elements that are arrayrefs and
hashrefs
str2array($string)
: convert string to hash using unescaping and
splitting on '&', supports elements that are arrayrefs and hashrefs
These routines allow one to make log messages in the lonnet.log and lonnet.perm logfiles.
logtouch()
: make sure the logfile, lonnet.log, exists
logthis()
: append message to the normal lonnet.log file, it gets
preiodically rolled over and deleted.
logperm()
: append a permanent message to lonnet.perm.log, this log
file never gets deleted by any automated portion of the system, only
messages of critical importance should go in here.
getfile($file,$caller)
: two cases - requests for files in /res or in /uploaded.
(a) files in /uploaded
(i) If a local copy of the file exists -
compares modification date of local copy with last-modified date for
definitive version stored on home server for course. If local copy is
stale, requests a new version from the home server and stores it.
If the original has been removed from the home server, then local copy
is unlinked.
(ii) If local copy does not exist -
requests the file from the home server and stores it.
If $caller is 'uploadrep': This indicates a call from lonuploadrep.pm (PerlHeaderParserHandler phase) for request for files originally uploaded via DOCS. - returns 'ok' if fresh local copy now available, -1 otherwise.
Otherwise: This indicates a call from the content generation phase of the request. - returns the entire contents of the file or -1.
(b) files in /res - returns the entire contents of a file or -1; it properly subscribes to and replicates the file if neccessary.
stat_file($url)
: $url is expected to be a /res/ or /uploaded/ style file
reference
returns either a stat()
list of data about the file or an empty list
if the file doesn't exist or couldn't find out about it (connection
problems or user unknown)
filelocation($dir,$file)
: returns file system location of a file
based on URI; meant to be ``fairly clean'' absolute reference, $dir is a
directory that relative $file lookups are to looked in ($dir of /a/dir
and a file of ../bob will become /a/bob)
hreflocation($dir,$file)
: returns file system location or a URL; same as
filelocation except for hrefs
declutter()
: declutters URLs (remove docroot, beginning slashes, 'res' etc)
userfileupload(): main rotine for putting a file in a user or course's filespace, arguments are,
formname - required - this is the name of the element in $env where the filename, and the contents of the file to create/modifed exist the filename is in $env{'form.'.$formname.'.filename'} and the contents of the file is located in $env{'form.'.$formname} context - if coursedoc, store the file in the course of the active role of the current user; if 'existingfile': store in 'overwrites' in /home/httpd/perl/tmp if 'canceloverwrite': delete file in tmp/overwrites directory subdir - required - subdirectory to put the file in under ../userfiles/ if undefined, it will be placed in "unknown"
(This routine calls clean_filename() to remove any dangerous characters from the filename, and then calls finuserfileupload() to complete the transaction)
returns either the url of the uploaded file (/uploaded/....) if successful and /adm/notfound.html if unsuccessful
clean_filename(): routine for cleaing a filename up for storage in userfile space, argument is:
filename - proposed filename
returns: the new clean filename
finishuserfileupload(): routine that creates and sends the file to userspace, probably shouldn't be called directly
docuname: username or courseid of destination for the file docudom: domain of user/course of destination for the file formname: same as for userfileupload() fname: filename (including subdirectories) for the file parser: if 'parse', will parse (html) file to extract references to objects, links etc. allfiles: reference to hash used to store objects found by parser codebase: reference to hash used for codebases of java objects found by parser thumbwidth: width (pixels) of thumbnail to be created for uploaded image thumbheight: height (pixels) of thumbnail to be created for uploaded image resizewidth: width to be used to resize image using resizeImage from ImageMagick resizeheight: height to be used to resize image using resizeImage from ImageMagick context: if 'overwrite', will move the uploaded file from its temporary location to userfiles to facilitate overwriting a previously uploaded file with same name. mimetype: reference to scalar to accommodate mime type determined from File::MMagic if $parser = parse.
returns either the url of the uploaded file (/uploaded/....) if successful and /adm/notfound.html if unsuccessful (or an error message if context was 'overwrite').
renameuserfile(): renames an existing userfile to a new name
Args: docuname: username or courseid of destination for the file docudom: domain of user/course of destination for the file old: current file name (including any subdirs under userfiles) new: desired file name (including any subdirs under userfiles)
mkdiruserfile(): creates a directory is a userfiles dir
Args: docuname: username or courseid of destination for the file docudom: domain of user/course of destination for the file dir: dir to create (including any subdirs under userfiles)
removeuserfile(): removes a file that exists in userfiles
Args: docuname: username or courseid of destination for the file docudom: domain of user/course of destination for the file fname: filname to delete (including any subdirs under userfiles)
removeuploadedurl(): convience function for removeuserfile()
Args: url: a full /uploaded/... url to delete
get_portfile_permissions(): Args: domain: domain of user or course contain the portfolio files user: name of user or num of course contain the portfolio files Returns: hashref of a dump of the proper file_permissions.db
get_access_controls():
Args:
current_permissions: the hash ref returned from get_portfile_permissions()
group: (optional) the group you want the files associated with
file: (optional) the file you want access info on
Returns: a hash (keys are file names) of hashes containing keys are: path to file/file_name\0uniqueID:scope_end_start (see below) values are XML containing access control settings (see below)
Internal notes:
access controls are stored in file_permissions.db as key=value pairs. key -> path to file/file_name\0uniqueID:scope_end_start where scope -> public,guest,course,group,domains or users. end -> UNIX time for end of access (0 -> no end date) start -> UNIX time for start of access
value -> XML description of access control <scope type=""> (type =1 of: public,guest,course,group,domains,users"> <start></start> <end></end>
<password></password> for scope type = guest
<domain></domain> for scope type = course or group <number></number> <roles id=""> <role></role> <access></access> <section></section> <group></group> </roles>
<dom></dom> for scope type = domains
<users> for scope type = users <user> <uname></uname> <udom></udom> </user> </users> </scope> Access data is also aggregated for each file in an additional key=value pair: key -> path to file/file_name\0accesscontrol value -> reference to hash hash contains key = value pairs where key = uniqueID:scope_end_start value = UNIX time record was last updated
Used to improve speed of look-ups of access controls for each file. Locks on files (resulting from submission of portfolio file to a homework problem stored in array of arrays.
modify_access_controls():
Modifies access controls for a portfolio file Args 1. file name 2. reference to hash of required changes, 3. domain 4. username where domain,username are the domain of the portfolio owner (either a user or a course)
Returns: 1. result of additions or updates ('ok' or 'error', with error message). 2. result of deletions ('ok' or 'error', with error message). 3. reference to hash of any new or updated access controls. 4. reference to hash used to map incoming IDs to uniqueIDs assigned to control. key = integer (inbound ID) value = uniqueID
escape()
: unpack non-word characters into CGI-compatible hex codes
unescape()
: pack CGI-compatible hex codes into actual non-word ASCII character
subreply()
: tries to pass a message to lonc, returns con_lost if incapable
reply()
: uses subreply to send a message to remote machine, logs all failures
critical()
: passes a critical message to another server; if cannot
get through then place message in connection buffer directory and
returns con_delayed, if incapable of saving message, returns
con_failed
reconlonc()
: tries to reconnect lonc client processes.
flushcourselogs()
: flush (save) buffer logs and access logs
courselog($what)
: save message for course in hash
courseacclog($what)
: save message for course using &courselog(). Perform
special processing for specific resource types (problems, exams, quizzes, etc).
goodbye()
: flush course logs and log shutting down; it is called in srm.conf
as a PerlChildExitHandler
symblist($mapname,%newhash)
: update symbolic storage links