nodes: 0, module; 1, function_definition; 2, function_name:lazy_imap; 3, parameters; 4, block; 5, identifier:data_processor; 6, identifier:data_generator; 7, default_parameter; 8, default_parameter; 9, expression_statement; 10, if_statement; 11, if_statement; 12, with_statement; 13, identifier:n_cpus; 14, integer:1; 15, identifier:stepsize; 16, None; 17, comment:"""A variant of multiprocessing.Pool.imap that supports lazy evaluation
  
      As with the regular multiprocessing.Pool.imap, the processes are spawned
      off asynchronously while the results are returned in order. In contrast to
      multiprocessing.Pool.imap, the iterator (here: data_generator) is not
      consumed at once but evaluated lazily which is useful if the iterator
      (for example, a generator) contains objects with a large memory footprint.
  
      Parameters
      ==========
      data_processor : func
          A processing function that is applied to objects in `data_generator`
  
      data_generator : iterator or generator
          A python iterator or generator that yields objects to be fed into the
          `data_processor` function for processing.
  
      n_cpus=1 : int (default: 1)
          Number of processes to run in parallel.
          - If `n_cpus` > 0, the specified number of CPUs will be used.
          - If `n_cpus=0`, all available CPUs will be used.
          - If `n_cpus` < 0, all available CPUs - `n_cpus` will be used.
  
      stepsize : int or None (default: None)
          The number of items to fetch from the iterator to pass on to the
          workers at a time.
          If `stepsize=None` (default), the stepsize size will
          be set equal to `n_cpus`.
  
      Returns
      =========
      list : A Python list containing the *n* results returned
          by the `data_processor` function when called on
          elements by the `data_generator` in
          sorted order; *n* is equal to the size of `stepsize`. If `stepsize`
          is None, *n* is equal to `n_cpus`.
      """; 18, not_operator; 19, block; 20, elif_clause; 21, comparison_operator:stepsize is None; 22, block; 23, with_clause; 24, block; 25, identifier:n_cpus; 26, expression_statement; 27, comparison_operator:n_cpus < 0; 28, block; 29, identifier:stepsize; 30, None; 31, expression_statement; 32, with_item; 33, while_statement; 34, assignment; 35, identifier:n_cpus; 36, integer:0; 37, expression_statement; 38, assignment; 39, as_pattern; 40, True; 41, block; 42, identifier:n_cpus; 43, call; 44, assignment; 45, identifier:stepsize; 46, identifier:n_cpus; 47, call; 48, as_pattern_target; 49, expression_statement; 50, if_statement; 51, attribute; 52, argument_list; 53, identifier:n_cpus; 54, binary_operator:mp.cpu_count() - n_cpus; 55, attribute; 56, argument_list; 57, identifier:p; 58, assignment; 59, identifier:r; 60, block; 61, else_clause; 62, identifier:mp; 63, identifier:cpu_count; 64, call; 65, identifier:n_cpus; 66, identifier:mp; 67, identifier:Pool; 68, keyword_argument; 69, identifier:r; 70, call; 71, expression_statement; 72, block; 73, attribute; 74, argument_list; 75, identifier:processes; 76, identifier:n_cpus; 77, attribute; 78, argument_list; 79, yield; 80, break_statement; 81, identifier:mp; 82, identifier:cpu_count; 83, identifier:p; 84, identifier:map; 85, identifier:data_processor; 86, call; 87, identifier:r; 88, identifier:islice; 89, argument_list; 90, identifier:data_generator; 91, identifier:stepsize
code: def lazy_imap(data_processor, data_generator, n_cpus=1, stepsize=None):
      """A variant of multiprocessing.Pool.imap that supports lazy evaluation
  
      As with the regular multiprocessing.Pool.imap, the processes are spawned
      off asynchronously while the results are returned in order. In contrast to
      multiprocessing.Pool.imap, the iterator (here: data_generator) is not
      consumed at once but evaluated lazily which is useful if the iterator
      (for example, a generator) contains objects with a large memory footprint.
  
      Parameters
      ==========
      data_processor : func
          A processing function that is applied to objects in `data_generator`
  
      data_generator : iterator or generator
          A python iterator or generator that yields objects to be fed into the
          `data_processor` function for processing.
  
      n_cpus=1 : int (default: 1)
          Number of processes to run in parallel.
          - If `n_cpus` > 0, the specified number of CPUs will be used.
          - If `n_cpus=0`, all available CPUs will be used.
          - If `n_cpus` < 0, all available CPUs - `n_cpus` will be used.
  
      stepsize : int or None (default: None)
          The number of items to fetch from the iterator to pass on to the
          workers at a time.
          If `stepsize=None` (default), the stepsize size will
          be set equal to `n_cpus`.
  
      Returns
      =========
      list : A Python list containing the *n* results returned
          by the `data_processor` function when called on
          elements by the `data_generator` in
          sorted order; *n* is equal to the size of `stepsize`. If `stepsize`
          is None, *n* is equal to `n_cpus`.
      """
      if not n_cpus:
          n_cpus = mp.cpu_count()
      elif n_cpus < 0:
          n_cpus = mp.cpu_count() - n_cpus
  
      if stepsize is None:
          stepsize = n_cpus
  
      with mp.Pool(processes=n_cpus) as p:
          while True:
              r = p.map(data_processor, islice(data_generator, stepsize))
              if r:
                  yield r
              else:
                  break