rlm_perl multiple processes/threads behavior

[Nick Rogers]

Hello,

I am hoping someone can clarify some questions I have about how rlm_perl
should behave in FreeRADIUS 2.2.5 with respect to multiple instances and
freeradius threads. I know there has been some discussion and changes/fixes
to this behavior over the years, and I've been using rlm_perl successfully
for nearly the last decade. Previously rlm_perl had its own "thread" pool,
where multiple instances of the perl process would run simultaneously. Now
I understand that rlm_perl uses the same freeradius thread management that
everything else does.

My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to be
single threaded, which has begun to cause performance issues at some of my
larger sites with increased load.

I do not have perl with multiplicity or ithreads compiled, as this causes
issues with other perl applications running on the same server. My question
is, is there a way to support multiple instances of the rlm_perl
interpreter without having ithreads or multiplicity compiled with perl?

Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.

Thanks!

-Nick

[Alan DeKok]

Now I understand that rlm_perl uses the same
freeradius thread management that everything else does.

Yes. It's simpler, and doesn't have problems.

My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to
be single threaded, which has begun to cause performance issues at some
of my larger sites with increased load.

It's more that the module used to spawn multiple interpretors. This
means extra overhead, etc.

I do not have perl with multiplicity or ithreads compiled, as this
causes issues with other perl applications running on the same server.
My question is, is there a way to support multiple instances of the
rlm_perl interpreter without having ithreads or multiplicity compiled
with perl?

One way is to build Perl libraries specifically for FreeRADIUS.

Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.

I don't think it ever spawned multiple Perl *processes*. It always
used just one.

You may be able to convert the module back to using a separate
interpretor per thread. I'm not familiar with Perl programming, so I
can't say more.

Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Nick Rogers]

Post by Alan DeKok

Now I understand that rlm_perl uses the same
freeradius thread management that everything else does.

Yes. It's simpler, and doesn't have problems.

My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to
be single threaded, which has begun to cause performance issues at some
of my larger sites with increased load.

It's more that the module used to spawn multiple interpretors. This
means extra overhead, etc.

I do not have perl with multiplicity or ithreads compiled, as this
causes issues with other perl applications running on the same server.
My question is, is there a way to support multiple instances of the
rlm_perl interpreter without having ithreads or multiplicity compiled
with perl?

One way is to build Perl libraries specifically for FreeRADIUS.

Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.

I don't think it ever spawned multiple Perl *processes*. It always
used just one.
You may be able to convert the module back to using a separate
interpretor per thread. I'm not familiar with Perl programming, so I
can't say more.

Thanks for the clarification and insights!

Post by Alan DeKok
Alan DeKok.
-
List info/subscribe/unsubscribe? See
http://www.freeradius.org/list/users.html

[Iliya Peregoudov]

rlm_perl had never spawn perl processes. rlm_perl embeds perl into the
radiusd process space, so no inter-process communication is needed to
call perl functions.

rlm_perl in freeradius 1.1 uses pool of interpreters. radiusd thread
take first unused interpreter from pool when rlm_perl is entered and put
the interpreter back to the pool when rlm_perl is leaved.

rlm_perl in freeradius 2.1 or higher uses interpreter per thread model.
When the thread enters the rlm_perl for first time the interpreter is
allocated and tied to the thread.

Perl interpreter allocation is only used when perl library is compiled
with USE_ITHREADS. If no USE_ITHREADS, then module is marked as thread
unsafe when built. freeradius core ensures not to call rlm_perl instance
in multiple threads simultaneously and rlm_perl instance uses single
interpreter.

If you want single rlm_perl instance to be called simultaneously from
different thread you need perl built with USE_ITHREADS.

However you can always call multiple rlm_perl instances simultaneously.
Define multiple perl instances with same script, and call them from
load-balance group:

module pl1 {
module = ${confdir}/pl.pl
}
module pl2 {
module = ${confdir}/pl.pl
}
module pl3 {
module = ${confdir}/pl.pl
}
module pl4 {
module = ${confdir}/pl.pl
}

authorize {
load-balance {
pl1 # 25% of requests go here
pl2
pl3
pl4
}
}

Post by Nick Rogers
Hello,
I am hoping someone can clarify some questions I have about how rlm_perl
should behave in FreeRADIUS 2.2.5 with respect to multiple instances and
freeradius threads. I know there has been some discussion and
changes/fixes to this behavior over the years, and I've been using
rlm_perl successfully for nearly the last decade. Previously rlm_perl
had its own "thread" pool, where multiple instances of the perl process
would run simultaneously. Now I understand that rlm_perl uses the same
freeradius thread management that everything else does.
My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to
be single threaded, which has begun to cause performance issues at some
of my larger sites with increased load.
I do not have perl with multiplicity or ithreads compiled, as this
causes issues with other perl applications running on the same server.
My question is, is there a way to support multiple instances of the
rlm_perl interpreter without having ithreads or multiplicity compiled
with perl?
Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.
Thanks!
-Nick

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Nick Rogers]

Post by Iliya Peregoudov
rlm_perl had never spawn perl processes. rlm_perl embeds perl into the
radiusd process space, so no inter-process communication is needed to call
perl functions.

Thanks for clarifying.

Post by Iliya Peregoudov
rlm_perl in freeradius 1.1 uses pool of interpreters. radiusd thread take
first unused interpreter from pool when rlm_perl is entered and put the
interpreter back to the pool when rlm_perl is leaved.
rlm_perl in freeradius 2.1 or higher uses interpreter per thread model.
When the thread enters the rlm_perl for first time the interpreter is
allocated and tied to the thread.
Perl interpreter allocation is only used when perl library is compiled
with USE_ITHREADS. If no USE_ITHREADS, then module is marked as thread
unsafe when built. freeradius core ensures not to call rlm_perl instance in
multiple threads simultaneously and rlm_perl instance uses single
interpreter.

Is MULTIPLICITY still necessary?

Is the CLONE function still relevant at all?

Post by Iliya Peregoudov
If you want single rlm_perl instance to be called simultaneously from
different thread you need perl built with USE_ITHREADS.
However you can always call multiple rlm_perl instances simultaneously.
Define multiple perl instances with same script, and call them from
This sounds like a good alternative for me and something I was thinking

about trying to figure out. I was expecting to have to build out multiple
virtual servers somehow and load balance between them. I did not realize
you could simply load balance between rlm_perl modules. Thanks!

Post by Iliya Peregoudov
module pl1 {
module = ${confdir}/pl.pl
}
module pl2 {
module = ${confdir}/pl.pl
}
module pl3 {
module = ${confdir}/pl.pl
}
module pl4 {
module = ${confdir}/pl.pl
}
authorize {
load-balance {
pl1 # 25% of requests go here
pl2
pl3
pl4
}
}

Post by Nick Rogers
Hello,
I am hoping someone can clarify some questions I have about how rlm_perl
should behave in FreeRADIUS 2.2.5 with respect to multiple instances and
freeradius threads. I know there has been some discussion and
changes/fixes to this behavior over the years, and I've been using
rlm_perl successfully for nearly the last decade. Previously rlm_perl
had its own "thread" pool, where multiple instances of the perl process
would run simultaneously. Now I understand that rlm_perl uses the same
freeradius thread management that everything else does.
My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to
be single threaded, which has begun to cause performance issues at some
of my larger sites with increased load.
I do not have perl with multiplicity or ithreads compiled, as this
causes issues with other perl applications running on the same server.
My question is, is there a way to support multiple instances of the
rlm_perl interpreter without having ithreads or multiplicity compiled
with perl?
Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.
Thanks!
-Nick

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/
list/users.html

[Iliya Peregoudov]

Post by Nick Rogers
Is MULTIPLICITY still necessary?

MULTIPLICITY is still necessary for creating multiple interpreters in
single process using perl_alloc().

Post by Nick Rogers
Is the CLONE function still relevant at all?

perl_clone() and CLONE are only relevant with USE_ITHREADS. Interpreters
created by perl_clone() can share variables. Interpreters created with
perl_alloc() cannot share variables.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Nick Rogers]

Post by Iliya Peregoudov
rlm_perl had never spawn perl processes. rlm_perl embeds perl into the
radiusd process space, so no inter-process communication is needed to call
perl functions.
rlm_perl in freeradius 1.1 uses pool of interpreters. radiusd thread take
first unused interpreter from pool when rlm_perl is entered and put the
interpreter back to the pool when rlm_perl is leaved.
rlm_perl in freeradius 2.1 or higher uses interpreter per thread model.
When the thread enters the rlm_perl for first time the interpreter is
allocated and tied to the thread.
Perl interpreter allocation is only used when perl library is compiled
with USE_ITHREADS. If no USE_ITHREADS, then module is marked as thread
unsafe when built. freeradius core ensures not to call rlm_perl instance in
multiple threads simultaneously and rlm_perl instance uses single
interpreter.
If you want single rlm_perl instance to be called simultaneously from
different thread you need perl built with USE_ITHREADS.
However you can always call multiple rlm_perl instances simultaneously.
Define multiple perl instances with same script, and call them from
module pl1 {
module = ${confdir}/pl.pl
}
module pl2 {
module = ${confdir}/pl.pl
}
module pl3 {
module = ${confdir}/pl.pl
}
module pl4 {
module = ${confdir}/pl.pl
}

I'm having trouble getting the above syntax to work. I ended up trying out
something like...

modules {
perl perl1 {
module = ${confdir}/perlscript
}
perl perl2 {
module = ${confdir}/perlscript2
}
}

accounting {
load-balance {
perl1
perl2
}
}

Both modules are initialized, but as far as I can tell only one perl
interpreter is being used, using the second script/module perl2. The load
balancing works, in that module perl1 and perl2 are called, but it seems
like the two modules are using the same interpreter/script, where the
last-defined perl module wins.

When I shutdown the server, I get strange messages indicating that the
second script was loaded on top of the first one in the same interpreter.

Attempt to free non-existent shared string 'is_ERROR' during global
destruction.


Is this because I do not have multiplicity compiled? Or should I be able to
have two unique instances of rlm_perl module with separate interpreters
that do not conflict?
Any suggestions? I'm using 2.2.0

Thanks

Post by Iliya Peregoudov
authorize {
load-balance {
pl1 # 25% of requests go here
pl2
pl3
pl4
}
}

Post by Nick Rogers
Hello,
I am hoping someone can clarify some questions I have about how rlm_perl
should behave in FreeRADIUS 2.2.5 with respect to multiple instances and
freeradius threads. I know there has been some discussion and
changes/fixes to this behavior over the years, and I've been using
rlm_perl successfully for nearly the last decade. Previously rlm_perl
had its own "thread" pool, where multiple instances of the perl process
would run simultaneously. Now I understand that rlm_perl uses the same
freeradius thread management that everything else does.
My problem is that my freeradius rlm_perl installation no longer spawns
multiple instances of the perl process, effectively causing my server to
be single threaded, which has begun to cause performance issues at some
of my larger sites with increased load.
I do not have perl with multiplicity or ithreads compiled, as this
causes issues with other perl applications running on the same server.
My question is, is there a way to support multiple instances of the
rlm_perl interpreter without having ithreads or multiplicity compiled
with perl?
Is there a way for freeradius to simply launch a perl process for every
freeradius server thread, instead of all threads relying on the same
rlm_perl process? I am not interested in using perl threads or having
multiple perl interpreters within the same perl process. I would really
appreciate some clarification if this is possible or not.
Thanks!
-Nick

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/
list/users.html

[Iliya Peregoudov]

Post by Nick Rogers
Is this because I do not have multiplicity compiled? Or should I be able
to have two unique instances of rlm_perl module with separate
interpreters that do not conflict?

Without MULTIPLICITY you cannot use more then one interpreter in a process.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html