JobBuilderReduceTKey, TValue, TOutput Method (IOperationInput, ActionTKey, IEnumerableTValue, RecordWriterTOutput, TaskContext, RecordReuseMode) |
Namespace: Ookii.Jumbo.Jet.Jobs.Builder
public StageOperation Reduce<TKey, TValue, TOutput>( IOperationInput input, Action<TKey, IEnumerable<TValue>, RecordWriter<TOutput>, TaskContext> reducer, RecordReuseMode recordReuse = RecordReuseMode.Default ) where TKey : Object, IComparable<TKey>
Note |
---|
Reduce tasks require that their input is already grouped by key. The JobBuilder class doesn't guarantee or verify this. To group the records in the same way other common MapReduce implementation do, use the SpillSortCombine(IOperationInput, Type, Type) function. |
Note |
---|
There is no non-delegate version of this method. To use an existing map task class, simply use the Process(IOperationInput, Type) function. |
This method generates a class inheriting from ReduceTaskTKey, TValue, TOutput which calls the target method of the reducer delegate from the Reduce(TKey, IEnumerableTValue, RecordWriterTOutput) method.
Note |
---|
The task method will be called from a completely different process than the one that is using JobBuilder, so it should not really on any external state. |
If recordReuse is Default and the target method has the AllowRecordReuseAttribute attribute applied to it, that attribute will be copied to the task class. If the target method has the ProcessAllInputPartitionsAttribute attribute applied to it, that attribute will be copied to the task class.
If the target method is a public method, it will be called directly by the generated task class. Otherwise, the supplied delegate will be serialized to the task settings and used to call the method. If the target method is an instance method, the instance it belongs to will be serialized as well (this class must have the SerializableAttribute attribute).