iteratee 0.3, 0.4 plans, pending patches

[John Lato]

Moving this discussion to the list as it pertains to pending patches.

The plan is to add these patches to the 0.3 line, upload a new version to
hackage, then include them in the 0.4 release.  I believe that hackage only
allows uploading of higher versions than what's already available, which is
the reason for this.  It shouldn't take much work at all to migrate them to
0.4.

On a related matter, does anyone have significant experience with
benchmarking?  I wouldn't mind somebody else taking a look at the benchmark
suite to check that it's sane.

Cheers,
John

On Wed, Aug 18, 2010 at 5:13 AM, Conrad Parker <
conrad.parker at tsurucapital.com> wrote:

> On 17 August 2010 19:25, John Lato <jwlato at gmail.com> wrote:
> > On Sat, Aug 14, 2010 at 2:00 AM, Conrad Parker
> > <conrad.parker at tsurucapital.com> wrote:
> >>
> >> On 13 August 2010 22:50, John Lato <jwlato at gmail.com> wrote:
> >> > Thanks very much; I'm traveling until Monday and won't be able to
> review
> >> > or
> >> > apply these until after I return, but I can confirm I've received
> these
> >> > patches.
> >>
> >> Hi John,
> >>
> >> thanks, I look forward to the feedback. I'm happy to work through any
> >> possible issues during the week. Usually I'd send patches day-by-day
> >> as work is done; with my own projects I don't really like it when
> >> someone comes and dumps a bunch of patches on me :) so I don't have
> >> any expectation that you'll simply accept everything as-is, and I do
> >> have time to work through any issues, add clarifiying
> >> documentation/examples etc.
> >
> > Hi Conrad,
> > I've looked through these patches and overall I'm very pleased with the
> > quality.  I do have a few questions/observations, and one minor issue I'd
> > like addressed.
>
> thanks for the feedback. I'll respond here briefly; perhaps we can
> follow up on the iteratees list though (or, I'm happy to just move
> this discussion there).
>
> > 1)  Have you benchmarked Data.Iteratee.Base.last ?  I'd like to know how
> it
> > performs relative to an implementation that operates chunk-wise rather
> than
> > element-wise.  If you use it and it's fast enough for you, I'm happy to
> add
> > it as-is.
>
> I'm happy using it as-is; at least in our application it is important
> to process an entire stream to update the state throughout.
> (Improving the stream format so that is not required is an unrelated
> issue ;-) Often we just use last to get the final state, eg.  as the
> first argument to enumFdFollow.
>
> > 2)  If I'm not mistaken, the key element of enumFdFollow is the added
> > threadDelay, yes?  IMO implementing this sort of thing is much simpler in
> > the 0.4 line because you can just use 'enumFromCallback'.
>
> yes, it replaces the return on eof with a threadDelay and retry. I
> couldn't see an easy way to compose this in 0.3. Follow seems to be a
> useful enough thing (at least for us ;-) that it could be worth
> providing an API call for anyway.
>
> In general, I'd like to discuss merging these functions into 0.3, then
> also help with porting them to work with 0.4.
>
> > 3)  The ioIter function looks great and the documentation is clear,
> however
> > I don't want to add a dependency on either monads-fd or monads-tf.
> >  Fortunately neither should be necessary if you import
> > Control.Monad.IO.Class (MonadIO, liftIO) instead of Control.Monad.Trans.
>
> Sounds good, I'll resend that patch (to the list :) with that change.
>
> > 4)  Have any of the INLINEs been benchmarked?  At least in the 0.4 line
> I've
> > found that INLINE'ing iteratees nearly always gives a substantial
> benefit,
> > however for enumerators and especially enumeratees it is much less
> > clear-cut.  For most enumeratees I've found that using SPECIALIZE gives
> the
> > best performance (sometimes by a significant margin).
> > Finally, do you have an opinion on ListLike?  Have you mostly used
> > ByteStrings for processing, or some other element type?  I would be most
> > interested to know what would be useful in terms of operations on data
> > chunks.
>
> We have been continuously benchmarking in an ad-hoc manner ;-) I
> expect you're right that the enumeratee INLINEs are less useful, and
> also on ioIter ...
>
> > Thanks very much for your submissions; I think they'll greatly increase
> the
> > usefulness and expressiveness of the package.
>
> great, thanks for the feedback :)
>
> Conrad.
>
> > Sincerely,
> > John
> >
> >>
> >> cheers,
> >>
> >> Conrad.
> >>
> >> >
> >> > Sincerely,
> >> > John
> >> >
> >> > On Tue, Aug 10, 2010 at 8:13 PM, Conrad Parker
> >> > <conrad.parker at tsurucapital.com> wrote:
> >> >>
> >> >> On 11 August 2010 07:39, Conrad Parker <
> conrad.parker at tsurucapital.com>
> >> >> wrote:
> >> >> > On 10 August 2010 18:42, John Lato <jwlato at gmail.com> wrote:
> >> >> >> Dear Conrad,
> >> >> >>
> >> >> >> I'm planning the initial iteratee-0.4.0 release for August 25.
>  It's
> >> >> >> been delayed as part of a testing effort, however I'm fairly
> >> >> >> confident
> >> >> >> in the code at this time so I feel ready to get it out.
> >> >> >
> >> >> > ok, great!
> >> >> >
> >> >> >> It's likely no surprise that I think the 0.4 line is much nicer
> than
> >> >> >> 0.3.x, but I plan to maintain 0.3.x for some time after the
> release
> >> >> >> and will accept patches for feature adds.  Migrating may be
> >> >> >> non-trivial, so I understand users may not be able to switch
> >> >> >> quickly.
> >> >> >> Of course a Hackage upload will be impossible without forking.
> >> >> >
> >> >> > I thought it was possible to simply upload tarballs with different
> >> >> > version numbers (eg. 0.3.6, then 0.4.0, then 0.3.7, 0.4.1)? The
> user
> >> >> > would have to explicitly install eg. iteratee-0.3.7 to get the 0.3
> >> >> > version though.
> >> >> >
> >> >> >>
> >> >> >> The additions you describe sound very useful, and I'd be most
> >> >> >> pleased
> >> >> >> to add them (the enumeratee generator sounds similar to
> >> >> >> "unfoldConvStream" in the 0.4 line, but the others are completely
> >> >> >> new).
> >> >> >
> >> >> > ok, I'll send some patches through to you.
> >> >>
> >> >> I've just sent through a bundle of 7 patches:
> >> >>
> >> >> Wed Aug 11 10:10:11 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * add convStateStream
> >> >>
> >> >> Wed Aug 11 09:27:48 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * add Data.Iteratee.IO.Interact
> >> >>
> >> >>  This introduces a dependency on monads-fd
> >> >>
> >> >> Wed Aug 11 09:26:45 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * add enumFollowFd, fileDriverFollowFd
> >> >>
> >> >> Wed Aug 11 08:03:25 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * add mapM_
> >> >>
> >> >> Wed Aug 11 08:03:05 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * Add last iteratee
> >> >>
> >> >> Wed Aug 11 07:34:16 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * update CONTRIBUTORS
> >> >>
> >> >> Wed Aug 11 07:33:36 JST 2010  Conrad Parker <conrad at metadecks.org>
> >> >>  * Revert pluralization of variants in Stream comment
> >> >>
> >> >> cheers,
> >> >>
> >> >> Conrad.
> >> >>
> >> >> >> Please let me know if you have any other feature requests or
> >> >> >> suggestions.
> >> >> >
> >> >> > Will do :)
> >> >> >
> >> >> > cheers,
> >> >> >
> >> >> > Conrad.
> >> >> >
> >> >> >>
> >> >> >> Best,
> >> >> >> John
> >> >> >>
> >> >> >> On Tue, Aug 10, 2010 at 1:57 AM, Conrad Parker
> >> >> >> <conrad.parker at tsurucapital.com> wrote:
> >> >> >>> Hi John,
> >> >> >>>
> >> >> >>> Over the last few weeks we've been developing some code using
> >> >> >>> iteratee-0.3.5 at Tsuru. In the course of this we've developed
> some
> >> >> >>> generic functions which would be nice to merge into the iteratee
> >> >> >>> library on Hackage. Are you still accepting patches for the 0.3.x
> >> >> >>> branch, or should we just target 0.4?
> >> >> >>>
> >> >> >>> The additions are:
> >> >> >>>
> >> >> >>>  * a "last" iteratee
> >> >> >>>  * an enumerator to follow a growing file descriptor (like tail
> -f)
> >> >> >>>  * an enumeratee generator (ie. a version of convStream) with
> >> >> >>> accumulated state and retention of remaining data from each Chunk
> >> >> >>>  * an iteratee for IO interaction (tested with both text commands
> >> >> >>> and
> >> >> >>> a GTK gui)
> >> >> >>>
> >> >> >>> also I'm working on some informal bloggy tutorials, but these
> >> >> >>> should
> >> >> >>> probably target the 0.4 branch. Do you have any general timeline
> >> >> >>> for
> >> >> >>> the 0.4 release?
> >> >> >>>
> >> >> >>> Conrad.
> >> >> >>>
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.haskell.org/pipermail/iteratee/attachments/20100818/25fef761/attachment.htm