Skip to content

More functools functionsΒ #55220

Closed
Closed

Description

@JasonBaker
BPO 11011
Nosy @rhettinger, @ncoghlan, @merwok, @bitdancer
Files
  • functools.patch: Patch for adding misc functions to functools
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2013-05-19.13:01:43.042>
    created_at = <Date 2011-01-25.22:32:21.638>
    labels = ['type-feature', 'library']
    title = 'More functools functions'
    updated_at = <Date 2013-10-04.09:34:57.239>
    user = 'https://bugs.python.org/JasonBaker'

    bugs.python.org fields:

    activity = <Date 2013-10-04.09:34:57.239>
    actor = 'piotr.dobrogost'
    assignee = 'none'
    closed = True
    closed_date = <Date 2013-05-19.13:01:43.042>
    closer = 'r.david.murray'
    components = ['Library (Lib)']
    creation = <Date 2011-01-25.22:32:21.638>
    creator = 'Jason.Baker'
    dependencies = []
    files = ['20520']
    hgrepos = []
    issue_num = 11011
    keywords = ['patch']
    message_count = 10.0
    messages = ['127062', '127066', '127067', '127076', '127085', '127088', '127102', '127128', '189046', '189585']
    nosy_count = 7.0
    nosy_names = ['rhettinger', 'ncoghlan', 'eric.araujo', 'r.david.murray', 'Jason.Baker', 'BreamoreBoy', 'piotr.dobrogost']
    pr_nums = []
    priority = 'normal'
    resolution = 'rejected'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue11011'
    versions = ['Python 3.3']

    Activity

    1. JasonBaker commented on Jan 25, 2011

      JasonBakermannequin
      MannequinAuthor

      I've created a patch that adds some common functional programming tools to functools. I've made the patch to work against Python 3.2, but that may be a bit aggressive. If so, then I can adapt it to work with 3.3.

      I also wouldn't be opposed to writing some of these in C if there's a performance need.

      The functions I added are:

      • flip - flip the first two arguments of a function
      • const - return a function that always returns the same thing
      • compose - compose multiple functions together
      • identity - returns what is passed in to it
      • trampoline - calls a function and then calls any returned functions.
    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Jan 25, 2011
    3. rhettinger commented on Jan 25, 2011

      @rhettinger
      Contributor

      Some of these have been proposed and rejected before.

      Compose has a problematic API because the traditional order of application in mathematics is counter-intuitive for many people.

      Const seems reasonable except that we already have ways to do it:
      twos = lambda :2
      twos = itertools.repeat(2).__next__
      The principal use case for const is with map() and itertools.repeat() is already targeted at that use-case:
      cubes = list(map(pow, range(10), repeat(3)))

      Flip would be nice on occasion:
      colorseq = list(map(flip, enumerate(
      'red orange yellow green blue indigo violet'.split())))

      Identity is problematic for a few reasons. First, there doesn't seem to be one signature that fits all use cases:
      identity1 = lambda *args: *args # always returns a tuple
      identity2 = lambda arg: arg # always returns a scalar
      Also, the availability of 'identity' seems to encourage bad design.
      Speedwise, it is usually better to have two paths (predicate is None vs some given predicate) than have an identity function default (predict=identity). See the itertools module for examples.

      Trampoline is interesting, but the use case doesn't seem to arise much in Python programming and when it does, it is usually clearer to write:
      for f in get_actions():
      f()
      That straight-forward code is superior, not just because of its readability but also because it is easily modified to handle return values being feed in to consecutive calls, adding optional and keyword arguments, introducing logging, etc.

      In short, some of these constructs are better suited for languages that a more purely functional in nature. Those don't treat scalar arguments differently than multiple arguments, those don't use keywords or optional arguments, currying can be done cleanly etc.

      Experiences with the itertools module have shown than not all of the usual favorite functional gizmos fit well in the Python language. They are tempting toys, but typically don't beat regular, clean Python code.

      One last comment, any time a group of interoperative tools is proposed, I think it absolutely necessary to construct a bunch of sample code to exercise the APIs working in concert with one another. With the itertools module, considerable effort was directed at designing a toolkit with cleanly interoperative parts.

      Also, when proposing something that changes the fundamentals of how people would design their programs, I think it is necessary to scan real-world code to find many examples where the code would be much improved if it used the new constructs. (This is similar to the notion of a Phase III clinical trial -- even if something works, it needs to be shown to be better than existing alternatives).

    4. rhettinger commented on Jan 25, 2011

      @rhettinger
      Contributor

      One other thought: The use of list comprehensions (a.k.a. list displays) and generator expressions has made many functional tools less necessary than ever.

      # preferred over map(pow, repeat(2), range(5))
      [pow(2, x) for x in range(5)]

      # preferred over map(flip, dict.items())
      [(v, k) for k, v in mydict.items()]

      The list comprehension and genexp styles are much more flexible and works well with default arguments, constants, and using a combination of features:

      # hard to do with functionals
      [f(g(2,x), h(x, 3, y, flag=True)) for x, y in zip(X, Y) if x>y//2]

    5. JasonBaker commented on Jan 26, 2011

      JasonBakermannequin
      MannequinAuthor

      Ray, thanks for prompt and thorough feedback. To address your concerns:

      • I'm fine with doing away with const and identity (long story short I haven't really used them in functional languages anyway). There were reasons for defining identity the way it is, but it's hardly a big deal if that doesn't make it in.
      • Personally, I see the mathematical notation for compose to be very intuitive, however it's typically described in a non-intuitive manner. The functions get called in a left-to-right manner (as is intuitive), however, when you see it defined it is usually in a right-to-left manner. Thus it can be confusing because we say compose(f, g)(x) [left-to-right] is equivalent to g(f(x)) [which reads right-to-left, but is still doing what you might expect intuitively]. But I doubt most people that use Python will care about the mathematical usage anyway. What if we called the function "pipeline"? That's essentially what it's doing, and I think people would find that easier to understand.
      • I'm not saying you're wrong about trampoline, but I want to make sure that you and I are discussing the same thing. :-) The idea for trampoline comes from clojure, which is unique among functional languages in that it has no tail-call recursion. Trampoline is a way to write a function that is tail recursive, but won't blow the stack. So your example, while similar isn't *quite* the same thing. The documentation I wrote shows an example function count_to_a_million. If I wrote it like this I'd blow the stack:
          def count_to_a_million(i):
              if i < 1000000:
                  return count_to_a_million(i)
              else:
                  return i

      Of course, you could definitely argue that it would just be easier to use a loop (or even better range). However, many functional programmers' brains have just been wired to use recursion which they might find more intuitive as a means of iteration. Of course, one could argue that it would be better for them to learn the Pythonic way of doing things, and I would agree. Python is a multi-paradigm language that supports both functional and imperative approaches, therefore it's Pythonic to provide a functional approach.

      • It sounds like you feel flip is useful, but it's a matter of whether you feel it's useful enough to add to the standard library.

      Lastly, I love list comprehensions and see where you're coming from, but why do the two have to be mutually exclusive? I mean, the idea for them came from Haskell which hasn't done away with them. For instance, take your example:

      [f(g(2,x), h(x, 3, y, flag=True)) for x, y in zip(X, Y) if x>y//2]
      

      I certainly hope you don't really write code like that. :-)

      I think something like this is more readable:

          fg = partial(compose(f, g), 2)
          h1 = lambda x, y:  h(x, 3, y, flag=True)
          [(fg(x), h1(x, y)) for x, y in zip(X, Y) if x>y//2]

      ...but maybe I'm weird.

      If you're still opposed to adding these functions, let me know. I feel that they have use-cases in the Python standard library, but I can add them to pysistence when I port it to Python 3 and make it a functional data structure/algorithm library. :-)

    6. ncoghlan commented on Jan 26, 2011

      @ncoghlan
      Contributor

      For flip, const and identity I agree there are already better ways to handle them using either itertools or comprehension syntax.

      The problem I have with trampoline is that it depends on the function's *return values* being defined in a certain way (i.e. returning a zero-argument callable instead of just calling them).

      The other major factor limiting the appeal of some of the more complex functional programming tools (the one that almost lead to lambda's removal in 3.x) is the availability of nested function definitions.

      To rephrase Raymond's last example in more idiomatic Python:

      def fg2(x):
        # Comment or docstring explaining fg2
        return f(g(2,x)
      
      def h2(x, y):
        # Comment or docstring explaining h2
        return h(x, 3, y, flag=True)
      
      result = [fg2(x), h2(x, y) for x, y in zip(X, Y) if x>y//2]

      I see compose as being the most potential useful, as it would allow us to provide a C accelerated version of the following alternative to trampoline:

      def compose(*args):
        def _composed(x):
          for f in args:
            x = f(x)
          return x
        return _composed

      While this actually does match the mathematical definition, I agree pointing that out is more confusing than helpful for many people (since the "inside-out" nested evaluation ends up looking like "right-to-left" evaluation when written down)

      Raymond's proposed alternative to trampoline could then be written:

      compose(*get_actions())(x)

    7. JasonBaker commented on Jan 26, 2011

      JasonBakermannequin
      MannequinAuthor

      I'm not sure I understand how Raymond's alternative for trampoline works. Let's take the factorial algorithm from wikipedia's page on tail recursion[1]. I've implemented the tail recursive version of the algorithm in Python using trampoline:

          from functools import trampoline, partial
      
          def factorial(n):
              def fact(i, acc):
                  if i:
                      return partial(fact, (i-1), (acc * i))
                  else:
                      return acc
              return trampoline(fact, n, 1)
          >>> factorial(5)
          120

      How would I implement this using Raymond's alternative?

      [1] http://en.wikipedia.org/wiki/Tail_call#Example_programs

    8. ncoghlan commented on Jan 26, 2011

      @ncoghlan
      Contributor

      How the conversion from a recursive algorithm to an iterative one works depends on the specific algorithm involved. A trampoline does the job for tail calls, but not necessarily any other recursive algorithm.

      Factorial actually has a fairly trivial iterative algorithm, so it isn't a great example for general purpose algorithm conversion:

      def factorial(x):
        result = 1
        for i in range(1, x+1):
          result *= i
        return result

      I believe having trampoline in functools would end up being something of an attractive nuisance - in cases where it applies, there are probably better, algorithm specific, ways of eliminating a recursive call.

    9. rhettinger commented on Jan 26, 2011

      @rhettinger
      Contributor

      I'm intrigued by the tampoline() but after reading Nick's post, I think it needs to be an external recipe or blog post with extensive examples so that it can mature and either prove its worth or serve as an intellectually stimulating toy.

    10. BreamoreBoy commented on May 12, 2013

      BreamoreBoymannequin
      Mannequin

      To summarize flip, const and identity won't happen, trampoline needs an external recipe or blog post and compose is the only one that's likely to happen. Opinions please gentlemen.

    11. bitdancer commented on May 19, 2013

      @bitdancer
      Member

      By my reading compose was also rejected, so I'm going to close this. (See bpo-1506122 for one previous rejection of compose.)

    12. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions