Sunday, January 27, 2013

When not to use Perl's Implicit close; Suffering from Buffering

This post is a quick not on a bug I had difficulty tracking down.

One nice feature of Perl, introduced long before my time, is that of implicit closing. Perl closes filehandles for you when you forget (maybe on purpose). So the following is not a resource leak as a standalone script:

open my $file, '>utf8', '/path/to/new/file'

    or die "couldn't open file: $!";

print $file 'Hello!';

When the script finishes, Perl will close $file for you, so you can be nice and lazy. The caveat to this is that the variable $. isn't reset as it would be with a normal close (see docs here). $. holds the current line number from the last file read. So if you were processing a file line-by-line and found an error, you might print an error like 'bad value foo on line XYZ' using the $. variable for XYZ. I raised a question about this on StackOverflow.

Today I found another case where not explicitly closing a filehandle means trouble. I was working on testing a modulino-style script with flexible outputs. You can call a method to set the handle that this script prints to. In my test script, I was setting the handle to be some filehandle and then checking the contents of the file against a string. The problem? The file was always empty at run time, but contained what I expected it to when I manually inspected it. Here's some example broken code:

package Demo::Bad::ImplicitClose;

use strict;

use warnings;

sub new {

 my ($class) = @_;

 my $self = {};

 bless $self, $class;

 return $self;


sub output_fh {

    my ( $self, $fh ) = @_;

    if ($fh) {

        if ( ref($fh) eq 'GLOB' ) {

            $self->{output_fh} = $fh;


        else {

            open my $fh2, '>', $fh or die "Couldn't open $fh";

            $self->{output_fh} = $fh2;





sub some_long_method {

 my ($self, $text) = @_;

 print { $self->{output_fh} } $text;



use strict;

use warnings;

use autodie;

use Test::More tests => 1;

use File::Slurp;

use Demo::Bad::ImplicitClose;

my $file_name = 'file1.txt';

#make sure we pass the test from outputting something *this* run

unlink $file_name if -e $file_name;

my $print = 'some junk';

my $demo = Demo::Bad::ImplicitClose->new();



my $contents = read_file($file_name);

is($contents, $print);

If you run, you'll see that its one and only test fails:

>perl -I[folder where you put the Demo directory]
not ok 1
#   Failed test at line 68.
#          got: ''
#     expected: 'some junk'
# Looks like you failed 1 test of 1.

Then, when you inspect the contents of file1.txt, you have:
some junk

What happened here? I was suffering from buffering. Because neither nor closed the file, it was still open when I was trying to read it. Nothing had been written to it yet because the amount printed was so small that it had to wait in the buffer either until there was more to write or until the file was closed, which would flush the buffer. Implicit close wouldn't be performed until the the filehandle's reference count reached 0, and the $demo object still had a reference to it. So the test would have worked fine if I had assigned undef to $demo, or just closed the filehandle.

Watch those implicit closes.

Sunday, January 20, 2013

Testing Perl Distributions with Test Subdirectories

Normally I run my test suites with the prove utility:
prove -vl --merge
The v option turns on verbose processing, and the l option adds lib/ to the include path. prove then runs all of the tests in the t/ folder.

Today, I had a new problem. I merged multiple distributions into one (without losing any Git history!), and each had a test suite that I wanted to keep separate. Naturally, I moved the tests from each distribution into its own subdirectory under t/. However, this time when I ran prove -vl, I got this message:

Files=0, Tests=0,  0 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)

Dubious... Well, I needed to know how to test with subdirectories in the t/ folder, so I looked at the prove documentation and found the -r option. The r stands for "recurse", meaning that the test files would be found by recursing into the directories of the distribution (starting at the top in the root of the distribution). That turned out to be exactly what I needed!
prove -vlr
All tests successful.
Files=28, Tests=1815, 211 wallclock secs ( 0.92 usr +  0.28 sys =  1.20 CPU)
Result: PASS


Also, both MakeMaker and Module::Build recurse in the same way during module testing. If you use Dist::Zilla, then you'll probably have the plugins [MakeMaker] and [ModuleBuild]. Using these, dzil test will recurse in the same way.