First of all, the executive summary: I think it'll be doable one way or another.

As Elian pointed out, if you want other people to understand what you're wanting, you might have to couch it in non-editor terms. (That being said, I used to hack around with Gosling's Emacs, so I know what you're saying.) Stated more generically, what you're asking for is a way to parameterize the lookup of variable names such that a particular generic variable name can refer to different specific variables at different times.

Perl has several mechanisms built in that get you most of the way there. As jdporter points out, package variables are thread local by default, so each thread gets its own copies of unshared variable. However, I don't think you necessarily want to tie a buffer to a thread.

As dakkar points out, you can get parameterized variables by using objects, where the attributes of the object can define themselves however they want. Perl 6 is trying to make object attributes work as much as possible like ordinary variables, so it might be acceptable to you to write $b.variable.

We've already said that you can curry classes into modules by assuming an invocant, so you could have a chunk of code that can even assume the $b. on the front. But then you don't get the sigils.

There's a mechanism built into regexes that variables of the form $?foo are kept in the current $0 object. That probably works outside of regexes as well.

Ordinary package variables don't support what you want, but if there were a way to declare a package using a global variable, then it'd be possible to compile variables in that scope to use that global variable as part of the variable's lookup strategy.

So there really are lots of ways to get the semantics you want. Which really means that you are asking for syntactic warpage after all. What you're asking for is that variables within a particular scope have a different lookup strategy than ordinary Perl names. So it'd be pretty easy to install a rule in the parser's grammar that lets you declare a variable using a new declarator that specifies a different "lifetime" for the variable. We currently have "my", "our", and "state" variables. You could set up a new declarator that declares buffer variables, and then binds the name however you jolly well please, using any of the parameterization methods we mentioned earlier to do the semantic warpage underneath.

If you want to discuss building a particular syntax in as standard, that should probably be discussed on the perl6-language mailing list.

I've wanted string bookmarks on several occasions in Perl. It is nice that you can parse with:

    if(   /.../gc    ) {
    } elsif(  /:::/gc  ) {
[download]

but it'd be even nicer to be able to work with substitutions in a similar manner, single-stepping different substitutions intermixed with different matches and being able to make multiple substr() LVALUE things that keep their position in the face of such substitutions or changes to other substr() LVALUE things.
                - tye

If you want any hope of getting this feature considered, you need to frame it much differently.

What I was really hoping was that there would be a way to implement it as a module. Looks like it will be (++premchai21), so all is well.

Perl isn't a text editor

No, it's a high-level language. I wasn't asking for Perl to be a text editor. (If I were, I'd have been asking for things like lowlevel keyboard access for checking when the buckies are held down.) We have Emacs already, and by the time Perl6 comes out we hope to have Emacs 22, which should be well on the way toward good enough to be considered as a text editor.

I'm not sure what prompted you to think I wanted Perl to be a text editor. Perhaps you saw "Emacs lisp" and just couldn't get "text editor" out of your head. In that case, let me just say that elisp is the language I use second-most (after Perl) precisely because it is the second-most-powerful language I know. It is, for example, more powerful than Python for many purposes. Buffers are a large part of the reason.

So maybe I should explain the power of buffers, and why I want to implement them in Perl6. Anyone who has ever programmed in elisp will already know most of this, but I guess this is perlmonks.org, not gnu.emacs.gnus, so here goes...

Buffers are like strings on steroids, except that that doesn't do their flexibility and usefulness justice. It's like saying "objects are structs on steroids". Buffers are not just a place to display text. In Emacs, display is a primary use of them, but even in Emacs not all buffers are intended to ever be shown to the user; various modules use them as workspaces for holding various kinds of state. Why don't they just use variables or arrays of variables? Because those structures are not as powerful. It's the same reason Perl programmers don't all store their paired data in arrays of arrays instead of hashes: hashes are more powerful.

What's so powerful about buffers? Well, in a word, they don't just hold data, but a significant amount of metadata as well. I'm not going to go into string properties (partly because they're not really only a feature of buffers; strings in elisp have them too), but I will talk about markers. These are the "string bookmarks" tye was talking about. Let's say you have some stuff stored in a buffer. Now, let's say you're working with it, and you've reached a certain point, and you want to hold your place there while you look at some other stuff. You set a marker (or save-excursion, which amounts to dynamically scoping the default marker (called point in elisp -- a lousy name and one I wouldn't keep when implementing this in Perl)), go look at the other data, and come back. No big deal, if it's just one, because there are other ways to do the same thing -- if it's just one spot. But now let's do something complex. Let's say, for example, that we want to implement an XML parser. (Yeah, I know it's been done in Perl5; it's been done in C and C++ too (think Gecko), but that doesn't mean a higher-level feature can't make it easier.) So, you've got your XML source in a buffer... now you want to create a DOM out of it. That means matching up tags, right?

Heh, heh, heh. This is where buffers get really cool. You pass through the buffer one time, setting a pair of markers for each tag you encounter (at the beginning and end of the tag), adding the tag to your array of tags, along with references to your markers indicating the tag's location. No need to worry about the data between them and where it fits just yet, because your markers point right there. Then you walk your array matching up the start tags with the end tags and putting references to them in a tree, and your DOM is complete. The markers send you straight back to the data. So far so good, but so far there are other ways to do it...

Now, here's what's really cool: let's say the data in the buffer changes; something is inserted (by, say, the DOM stuff an an ECMA script). Your markers all still point to the right places. Try *that* with a string and pointers. Not possible -- which is why parsers done in langauges without buffers end up with horrible hacks like pulling bits and pieces of the source out into little pieces as they parse it, storing the bits in little strings, and attaching those to the DOM tree. As a result, they either have a complete duplicate of all the data, or else they mangle and lose the original source (what Gecko does) and have to try to reconstruct it if it's needed again. Ick. If it had been implemented in a language with buffers and markers, things would have been much easier and more sane.

You could argue that what I've just described is all text-editing stuff, but in that case most of the major applications in existence -- web browsers, spreadsheets, databases (yes, buffers can hold binary data), word processors, web servers, mail servers, ... they're all text editors. Because they all handle text. Lots of it.

Yes, there are programs you could write that would derive small or no benefit from buffers. Photo editors (a la Gimp) come to mind. There are also programs that don't need hashes or objects, but that doesn't stop Perl from giving us useful things.

I've only scratched the very surface of the usefulness of buffers. They're at least as useful as objects or hashes. For a full explanation, I refer you to the Gnu Emacs Lisp Reference Manual. HTH.HAND.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
[download]

That all sounds great. So what's stopping you from building a module to implement buffers and putting it on CPAN right now? Surely it's possible to implement an elisp-style buffer system in Perl and/or C.

Update: I just reread your post and I see now that you're worried about localising global variables within buffers. Why? You don't mention anything about global variables in your explanation of why buffers would be great to have in Perl.

Perl 5 exposes its symbol tables and typeglobs to whatever manipulation your twisted mind can imagine. I'd be surprised if there was anything you really couldn't do with them!

-sam

You're still not making enough sense to make this workable. (And, like it or not, I'm the person who has to understand it--if I don't, you don't get it)

Emacs has the concept of a buffer and everything centers around it. That's fine. What perl construct are you proposing take its place? A class? A closure? A method? A file? Should it be tied to data and happen when that data is manipulated? If so, then what do you do when two pieces of data both override the same stuff?

Perl is not a text editor, and text editor metaphors don't always translate over. This would be one of those cases.

Could you have an on-buffer-change hook of some sort that accesses a data structure keeping track of which variables were buffer-local to which buffers and swaps their values in and out appropriately?

That is, something like (quite untested):

sub onBufferChange
{
  my ($from, $to) = @_;
  for my $scl (values $from->{locals}) {
    $$$scl[0] = $globalLocals{$$scl[0]}[1];
  }
  for my $scl (values $to->{locals}) {
    exists $globalLocals{$$scl[0]} or
      $globalLocals{$$scl[0]} = [$$scl[0], $$$scl[0]];
    $$$scl[0] = $$scl[1];
  }
}
[download]

I have the feeling I've missed the point...

++

I actually feel pretty stupid for not seeing that. It's quite simple, really: rather than making the current buffer a package variable on the buffer class, I just make it a method (that functions as an lvalue if need be; I'm pretty sure that's possible in Perl6) on the buffer class, and then I can do whatever symbol table changes I need to do. It makes changing buffers very frequently a little more inefficient, but it won't be a big deal unless you have a lot of buffer-local variables, and in that case it's worth it because you're really using the feature. And actually, it's probably not less efficient than making every variable lookup check for buffer-localness.

I'm happy. Perl6 is going to be so cool. Here's looking forward to the next Apocalypse :-)

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
[download]

Hmmm... prototype-based programming?

Something like Self (or NewtonScript), where your objects inherit not from classes but from prototypes (usually one, but it can be extended), and each object is more-or-less a hash (with subrefs for methods), and you search for a value, given the key, starting from the invocant (which is passed to each method call unchanged) up through the proto chain.

$obj1={
  _proto=>$base_obj,
  do_something=>sub{
    print $_[0]->get('string')},
  string=>'object one'};
$obj2={
  _proto=>$obj1,
  change=>sub{
    $_[0]->set('string',$_[1])}};

$obj2->call('do_something');
$obj2->call('set','object two');
$obj2->call('do_something');
$obj1->call('do_something');
__END__
would print:

object one
object two
object one
[download]

This would work since the get walks the proto chain, but set sets thing directly on the invocant.

Of course you can clean up the syntax using AUTOLOAD and tied hashes, but that's just sugar.

In your case, you'll have each buffer as an object, with proto=>$BaseBuffer, and each "mode" will add itself to the proto chain: on reading, the value will be the mode's default, on writing you'll get a "buffer-local" variable.

-- 
        dakkar - Mobilis in mobile

Hmmm... prototype-based programming?

E.g. Class::Classless and Class::Object.

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.

I don't understand WHY would you want the ever-changing globals. The whole idea of "current buffer" sounds a little silly to me. (No offence meant!) Why would you want to restrict yourself to one "current" buffer, having to "switch" buffers, having to remember in what part of the program the variables point to this buffer and when to the other? This is like having to use

select(OUT);
print "To the file\n";
select(STDOUT);
print "To the screen\n";
[download]

I think that your "strings on steroids" could be nice, but why don't you keep the "variables" as some properties/fields of the object? So you are going to have to use $obj->{varname} or $obj->{PROP}{varname} instead of $var, who cares? I would not want a single statement to change the values of tens of totaly unrelated variables that might have been at some point declared as "per-buffer".

I'd say forget the globals and start coding in Perl5. Maybe you'll find out that 1) the syntactic sugar is not that important and 2) that you may make most of it even in Perl5.

IMHO the biggest problem is that people will want to be able to change the text inside the buffer as easily as they do change strings. Eg. to be able to apply a s/regexp/relacement/g to the buffer. THAT will be a challenge with all those markers laying around.

Jenda
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
   -- Rick Osborne

I don't understand WHY would you want the ever-changing globals. The whole idea of "current buffer" sounds a little silly to me. (No offence meant!) Why would you want to restrict yourself to one "current" buffer, having to "switch" buffers, having to remember in what part of the program the variables point to this buffer and when to the other?

I can assure you that in practice it is never any problem to remember what buffer you are working with if it matters; *usually* it doesn't matter, because most of what you want to do you want to do in a buffer-agnostic fashion.

The advantage of buffer-local variables is much the same as the advantage of dynamically scoped ("local" in Perl5, temporized in Perl6) variables: you can customise another package's behavior when it works with your data, without worrying about the effects leaking out when a third package uses your stuff and also uses directly the package you're customising.

Let's say you're writing a CSS parser, for instance. Now, the CSS parser needs to hang the style information on a DOM, so your CSS parser is going to require a DOM package of some kind, of course. But you want the DOM package to behave in a certain way. So you temporize some of its package variables. But when someone else writes an HTML parser, he'll be using your CSS parser, but he'll _also_ want to use the DOM stuff. So he'll require both. But your changes to the DOM package stuff won't have an impact on his use of the DOM stuff. Now, all of that can be achieved with temp (local in Perl5), but when you start doing somewhat more complex things, it's handy if a single package can customize the behavior of another package in several different ways for various hunks of data. For example, an OpenOffice document parser might customise the XML parsing stuff differently for content.xml versus how it wants things done for styles.xml, or whatever. (This is a contrived example, because all the good already-implemented examples are Emacs stuff.)

The whole point, in fact, of buffer-localisation of variables is that you don't need to keep track of what buffer you're using when (and a sub can use different buffers at different times), because each buffer can carry its stuff (in this case, customisations of other packages' vars) along with it.

Yes, it could be done just by attaching a hash of customisations to each buffer, but then all the packages that you might want to customise would have to be aware of and pay attention to such things, which would be a major pain.

Now, all up to this point I've been talking about the semantics of make-local-variable, which simply gives a variable a buffer-local value associated with the current buffer; any time a different buffer is current, that variable will not be buffer-local. So, for example, if a closure foo has a private buffer (which is probably anonymous, but we'll call it foo_private_buffer) that it uses to store a bunch of state, and if it gives that buffer a buffer-local value for $Bar::Baz, then whenever any sub from package Bar is operating while foo_private_buffer is current it'll see foo's value of Baz. However, if foo calls a sub from package Twiddle, which uses a package-scoped buffer from package Twiddle, and if the the sub from Twiddle also calls a sub from Bar, _that_ sub will be working with (and changing, if applicable) the default (package-scoped) value of Baz.

However, elisp also has a mechanism (called make-variable-buffer-local) whereby a variable has a default value, but a normal attempt to change the value of the variable automatically makes it buffer-local to the current buffer, leaving the default unchanged. (The default can still be changed explicitely, however.) This semantic seems less useful to me, and I may not bother to implement it, though it would be easily enough added in if someone felt the need for it.

---

for(unpack("C*",'GGGG?GGGG?O__\?WccW?{GCw?Wcc{?Wcc~?Wcc{?~cc'
.'W?')){$j=$_-63;++$a;for$p(0..7){$h[$p][$a]=$j%2;$j/=2}}for$
p(0..7){for$a(1..45){$_=($h[$p-1][$a])?'#':' ';print}print$/}
[download]

I can assure you that the only reason why eLisp uses dynamic scoping instead of lexical is that it was designed before lexical scoping became common-place in the Lisp world, and nobody wanted to face the backwards-compatibility nightmare of changing it. If it had been invented a few years later, it would have used lexical scoping from the get-go. And this would have been a Good Thing.

So why repeat design decisions which are now thought to be mistakes?

OK, so let's stop discussing whether it's a good idea or not. We can't convince each other anyway. It seems to me that the whole thing you'd need is to be able to "localyze to an outer block". Basicaly you'd need to be able to localyze some variables in your SwitchBuffer() function, but the localyzation should hold to the end of the block enclosing the SwitchBuffer() CALL. Do I make sense?

I do believe there is some way to do this already, using some lower level trickery. Cause this looks it could be helpfull in other places as well. If I were you I'd wait a day or two and if noone suggests a module, try to ask again in the SoPW.

Jenda
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
   -- Rick Osborne

Edit by castaway: Closed small tag in signature

Aren't you are describing an inheritance tree for variables? So you can map this onto Perl semantics by assuming each thing seen as a variable is really a pair of setter/getters. The current buffer's variable container then simply inherits from the global variable container.

I can already half-imagine ways to do this with AUTOLOAD magic as I'm typing so as to automagically provide the setter/getter pairs you need. If you need arbitrarily nested buffers, it gets difficult in Perl5 as you'll have create packages on the fly and I'd generally stay away from that, but it shouldn't be hard in Perl6 if I'm not mistaken. If you don't, esp if you only need two levels, then it's fairly straightforward to implement even in Perl 5.

I can elaborate further if desired.

Makeshifts last the longest.

I almost got you what you were talking about ... and then I didn't. Your explanations are articulate, but I think you're talking from a place (a paradigm, dare I say it) that's a bit too foreign for us non-LISP people. How about if you wrote some code? That is, imagine some (hopefully small) example that uses this feature, without worrying about how it's implemented. And write it in "mostly-Perl", along with comments to help us see what's happening.

I wonder if tying variables, particularly scalars, could help in the implementation, but really I think I don't understand enough yet to comment usefully.