Click or drag to resize

JobBuilderReduceTKey, TValue, TOutput(IOperationInput, ActionTKey, IEnumerableTValue, RecordWriterTOutput, RecordReuseMode) Method

Runs a reduce function on each key in the specified input.

Namespace: Ookii.Jumbo.Jet.Jobs.Builder
Assembly: Ookii.Jumbo.Jet (in Ookii.Jumbo.Jet.dll) Version: 0.3.0+dc1307f20e065bb638e0b73a34cd216f57e486f1
Syntax
public StageOperation Reduce<TKey, TValue, TOutput>(
	IOperationInput input,
	Action<TKey, IEnumerable<TValue>, RecordWriter<TOutput>> reducer,
	RecordReuseMode recordReuse = RecordReuseMode.Default
)
where TKey : Object, IComparable<TKey>

Parameters

input  IOperationInput
The input.
reducer  ActionTKey, IEnumerableTValue, RecordWriterTOutput
The reducer function.
recordReuse  RecordReuseMode  (Optional)
The record reuse mode.

Type Parameters

TKey
The type of the keys.
TValue
The type of the values.
TOutput
The type of the output records.

Return Value

StageOperation
A StageOperation instance that can be used to further customize the operation.
Remarks
Note  Note
Reduce tasks require that their input is already grouped by key. The JobBuilder class doesn't guarantee or verify this. To group the records in the same way other common MapReduce implementation do, use the SpillSortCombine(IOperationInput, Type, Type) function.
Note  Note
There is no non-delegate version of this method. To use an existing map task class, simply use the Process(IOperationInput, Type) function.

This method generates a class inheriting from ReduceTaskTKey, TValue, TOutput which calls the target method of the reducer delegate from the Reduce(TKey, IEnumerableTValue, RecordWriterTOutput) method.

Note  Note
The task method will be called from a completely different process than the one that is using JobBuilder, so it should not really on any external state.

If recordReuse is Default and the target method has the AllowRecordReuseAttribute attribute applied to it, that attribute will be copied to the task class. If the target method has the ProcessAllInputPartitionsAttribute attribute applied to it, that attribute will be copied to the task class.

The target method must be a public static method.

See Also