[Rd] How to understand packages, namespaces, environments

[Duncan Murdoch]

Alexander Stoddard wrote:
> I would be very glad of pointers to information on how the concepts of 
> packages, namespaces and environments are interrelated in R.

I don't think that documentation exists yet, but I did update the 
documentation on environments for 2.1.0.  Look in the section (2.1.10 I 
think) on Environment Objects in the R language definition manual. 
Basically environments are things where you can ask to look up names and 
R will return R objects to you.  They're a bit more complicated than 
just a list of (name, object) pairs, but can (almost) be used as that 
(and I'm hoping will be able to be used that way in 2.2.0, but that 
remains to be seen).

> I am trying to get a handle on this both so I can delve further into 
> understanding other people's code and so I can organize my own in a more 
> coherent manner.
> 
>  From my reading about environments it seems they function as what I 
> would intuitively call namespaces. However, the documentation for the 
> 'library' function implies that "namespace" has a specific meaning in R 
> that I have so far failed to grasp. What is that meaning (or where 
> should I look to read up on it) ?

A namespace is a more abstract concept, which happens to be implemented 
in R using environments. Exactly how is a bit of a mystery to me, I'm 
afraid!  However, the basic idea is that a namespace defines a fixed set 
of meanings for names, which the code in a package can assume won't 
change.  It may also hide some names from outsiders.
> 
> I can think of the following more specific questions. Perhaps they may 
> most usefully serve to reveal my misconceptions, so corrections would be 
> very helpful.
> 
> Does saying the following load package 'foo' into its own environment ?
>  > library(foo)

This loads some of the (name, object) pairs from the package into two 
environments:  the public one the user can see, and the namespace one 
that the code in the package can see.  They're related, you don't get 
two copies of the objects.
> 
> Do environments have names?

They are R objects, so some of them have names, and some of them are 
anonymous parts of other objects.
> 
> Of what does the list returned by search() actually consist? Is it a 
> list of environments, a list that may include environments, or something 
> else?

It's a list of strings that can be used to get corresponding 
environments.  You don't get the actual environments.
> 
> What is actually designated by a character vector of form "package:foo"?

That string is used by the as.environment() function to go through the 
search list and obtain the environment.  For instance, you can do

x <- as.environment("package:foo")

and x will be a reference to the environment.  Then

ls(x)

will list all the objects there, get('bar', envir=x)  will extract the 
object named 'bar' from it, etc.
> 
> In what ways can I use the character vector "package:foo" when 
> interacting with R?

I can't think of any other uses than in as.environment(), but there 
might be some, or some automatic calls to it.  I'd have to check the man 
pages to be sure, but perhaps some functions that accept envir=x type 
arguments would automatically call as.environment on x first.

> Many thanks,
> Alex Stoddard
> 
> P.S. It took me a long time of flailing about to discover the 'search' 
> function. Perhaps it could be included in the "See Also" section of the 
> help for 'ls'.

That's a good suggestion.  Could you do me a favour, and as you're 
reading up on this stuff identify a whole list of deficiencies in the 
documentation and suggested fixes for them?  Making a change to a man 
page has a certain fixed overhead (making sure there are no syntax 
errors, committing, possibly porting the change to the patch branch), so 
batching them is efficient.

Duncan Murdoch