perlhacktips (1) Linux Manual Page

NAME

perlhacktips – Tips for Perl core C code hacking

DESCRIPTION

This document will help you learn the best way to go about hacking on the Perl core C code. It covers common problems, debugging, profiling, and more.

If you haven’t read perlhack and perlhacktut yet, you might want to do that first.

COMMON PROBLEMS

Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. You don’t care about some particular platform having broken Perl? I hear there is still a strong demand for J2EE programmers.

Perl environment problems

•

Not compiling with threading

Compiling with threading (-Duseithreads) completely rewrites the function prototypes of Perl. You better try your changes with that. Related to this is the difference between “Perl_-less” and “Perl_-ly” APIs, for example:

Perl_sv_setiv(aTHX_...);
sv_setiv(...);

The first one explicitly passes in the context, which is needed for e.g. threaded builds. The second one does that implicitly; do not get them mixed. If you are not passing in a aTHX_, you will need to do a dTHX (or a dVAR) as the first thing in the function.

See “How multiple interpreters and concurrency are supported” in perlguts for further discussion about context.

•

Not compiling with -DDEBUGGING

The DEBUGGING define exposes more code to the compiler, therefore more ways for things to go wrong. You should try it.

•

Introducing (non-read-only) globals

Do not introduce any modifiable globals, truly global or file static. They are bad form and complicate multithreading and other forms of concurrency. The right way is to introduce them as new interpreter variables, see intrpvar.h (at the very end for binary compatibility).

Introducing read-only (const) globals is okay, as long as you verify with e.g. "nm libperl.a|egrep -v ' [TURtr] '" (if your "nm" has BSD-style output) that the data you added really is read-only. (If it is, it shouldn’t show up in the output of that command.)

If you want to have static strings, make them constant:

static const char etc[] = “…”;

If you want to have arrays of constant strings, note carefully the right combination of "const"s:

static const char *const yippee[] =
{“hi”, “ho”, “silver”};

There is a way to completely hide any modifiable globals (they are all moved to heap), the compilation setting "-DPERL_GLOBAL_STRUCT_PRIVATE". It is not normally used, but can be used for testing, read more about it in “Background and PERL_IMPLICIT_CONTEXT” in perlguts.

•

Not exporting your new function

Some platforms (Win32, AIX, VMS, OS/2, to name a few) require any function that is part of the public API (the shared Perl library) to be explicitly marked as exported. See the discussion about embed.pl in perlguts.

•

Exporting your new function

The new shiny result of either genuine new functionality or your arduous refactoring is now ready and correctly exported. So what could possibly go wrong?

Maybe simply that your function did not need to be exported in the first place. Perl has a long and not so glorious history of exporting functions that it should not have.

If the function is used only inside one source code file, make it static. See the discussion about embed.pl in perlguts.

If the function is used across several files, but intended only for Perl’s internal use (and this should be the common case), do not export it to the public API. See the discussion about embed.pl in perlguts.

Portability problems

The following are common causes of compilation and/or execution failures, not common to Perl as such. The C FAQ is good bedtime reading. Please test your changes with as many C compilers and platforms as possible; we will, anyway, and it’s nice to save oneself from public embarrassment.

If using gcc, you can add the "-std=c89" option which will hopefully catch most of these unportabilities. (However it might also catch incompatibilities in your system’s header files.)

Use the Configure "-Dgccansipedantic" flag to enable the gcc "-ansi -pedantic" flags which enforce stricter ANSI rules.

If using the "gcc -Wall" note that not all the possible warnings (like "-Wuninitialized") are given unless you also compile with "-O".

Note that if using gcc, starting from Perl 5.9.5 the Perl core source code files (the ones at the top level of the source code distribution, but not e.g. the extensions under ext/) are automatically compiled with as many as possible of the "-std=c89", "-ansi", "-pedantic", and a selection of "-W" flags (see cflags.SH).

Also study perlport carefully to avoid any bad assumptions about the operating system, filesystems, character set, and so forth.

You may once in a while try a “make microperl” to see whether we can still compile Perl with just the bare minimum of interfaces. (See README.micro.)

Do not assume an operating system indicates a certain compiler.

•

Casting pointers to integers or casting integers to pointers

    void castaway(U8* p)
    {
      IV i = p;

or

void castaway(U8 *p)
{
IV i = (IV)p;

Both are bad, and broken, and unportable. Use the PTR2IV() macro that does it right. (Likewise, there are PTR2UV(), PTR2NV(), INT2PTR(), and NUM2PTR().)

•

Casting between function pointers and data pointers

Technically speaking casting between function pointers and data pointers is unportable and undefined, but practically speaking it seems to work, but you should use the FPTR2DPTR() and DPTR2FPTR() macros. Sometimes you can also play games with unions.

•

Assuming sizeof(int) == sizeof(long)

There are platforms where longs are 64 bits, and platforms where ints are 64 bits, and while we are out to shock you, even platforms where shorts are 64 bits. This is all legal according to the C standard. (In other words, “long long” is not a portable way to specify 64 bits, and “long long” is not even guaranteed to be any wider than “long”.)

Instead, use the definitions IV, UV, IVSIZE, I32SIZE, and so forth. Avoid things like I32 because they are not guaranteed to be exactly 32 bits, they are at least 32 bits, nor are they guaranteed to be int or long. If you really explicitly need 64-bit variables, use I64 and U64, but only if guarded by HAS_QUAD.

•

Assuming one can dereference any type of pointer for any type of data

char *p = …;
long pony = *(long *)p; /* BAD */

Many platforms, quite rightly so, will give you a core dump instead of a pony if the p happens not to be correctly aligned.

•

Lvalue casts

  (int)*p = ...;    /* BAD */

Simply not portable. Get your lvalue to be of the right type, or maybe use temporary variables, or dirty tricks with unions.

•

Assume anything about structs (especially the ones you don’t control, like the ones coming from the system headers)

•

That a certain field exists in a struct

•

That no other fields exist besides the ones you know of

•

That a field is of certain signedness, sizeof, or type

•

That the fields are in a certain order

•

While C guarantees the ordering specified in the struct definition, between different platforms the definitions might differ

•

That the sizeof(struct) or the alignments are the same everywhere

•

There might be padding bytes between the fields to align the fields – the bytes can be anything

•

Structs are required to be aligned to the maximum alignment required by the fields – which for native types is for usually equivalent to sizeof() of the field

•

Assuming the character set is ASCIIish

Perl can compile and run under EBCDIC platforms. See perlebcdic. This is transparent for the most part, but because the character sets differ, you shouldn’t use numeric (decimal, octal, nor hex) constants to refer to characters. You can safely say 'A', but not 0x41. You can safely say '
', but not "
". However, you can use macros defined in utf8.h to specify any code point portably. "LATIN1_TO_NATIVE(0xDF)" is going to be the code point that means LATIN SMALL LETTER SHARP S on whatever platform you are running on (on ASCII platforms it compiles without adding any extra code, so there is zero performance hit on those). The acceptable inputs to "LATIN1_TO_NATIVE" are from 0x00 through 0xFF. If your input isn’t guaranteed to be in that range, use "UNICODE_TO_NATIVE" instead. "NATIVE_TO_LATIN1" and "NATIVE_TO_UNICODE" translate the opposite direction.

If you need the string representation of a character that doesn’t have a mnemonic name in C, you should add it to the list in regen/unicode_constants.pl, and have Perl create "#define"‘s for you, based on the current platform.

Note that the "isFOO" and "toFOO" macros in handy.h work properly on native code points and strings.

Also, the range ‘A’ – ‘Z’ in ASCII is an unbroken sequence of 26 upper case alphabetic characters. That is not true in EBCDIC. Nor for ‘a’ to ‘z’. But ‘0’ – ‘9’ is an unbroken range in both systems. Don’t assume anything about other ranges. (Note that special handling of ranges in regular expression patterns and transliterations makes it appear to Perl code that the aforementioned ranges are all unbroken.)

Many of the comments in the existing code ignore the possibility of EBCDIC, and may be wrong therefore, even if the code works. This is actually a tribute to the successful transparent insertion of being able to handle EBCDIC without having to change pre-existing code.

UTF-8 and UTF-EBCDIC are two different encodings used to represent Unicode code points as sequences of bytes. Macros with the same names (but different definitions) in utf8.h and utfebcdic.h are used to allow the calling code to think that there is only one such encoding. This is almost always referred to as "utf8", but it means the EBCDIC version as well. Again, comments in the code may well be wrong even if the code itself is right. For example, the concept of UTF-8 "invariant characters" differs between ASCII and EBCDIC. On ASCII platforms, only characters that do not have the high-order bit set (i.e. whose ordinals are strict ASCII, 0 – 127) are invariant, and the documentation and comments in the code may assume that, often referring to something like, say, "hibit". The situation differs and is not so simple on EBCDIC machines, but as long as the code itself uses the "NATIVE_IS_INVARIANT()" macro appropriately, it works, even if the comments are wrong.

As noted in “TESTING” in perlhack, when writing test scripts, the file t/charset_tools.pl contains some helpful functions for writing tests valid on both ASCII and EBCDIC platforms. Sometimes, though, a test can’t use a function and it’s inconvenient to have different test versions depending on the platform. There are 20 code points that are the same in all 4 character sets currently recognized by Perl (the 3 EBCDIC code pages plus ISO 8859-1 (ASCII/Latin1)). These can be used in such tests, though there is a small possibility that Perl will become available in yet another character set, breaking your test. All but one of these code points are C0 control characters. The most significant controls that are the same are "