Click or drag to resize

JobBuilderReduceTKey, TValue, TOutput Method (IOperationInput, ActionTKey, IEnumerableTValue, RecordWriterTOutput, TaskContext, RecordReuseMode)

Runs a reduce function on each key in the specified input.

Namespace:  Ookii.Jumbo.Jet.Jobs.Builder
Assembly:  Ookii.Jumbo.Jet (in Ookii.Jumbo.Jet.dll) Version: 2.0.0
Syntax
public StageOperation Reduce<TKey, TValue, TOutput>(
	IOperationInput input,
	Action<TKey, IEnumerable<TValue>, RecordWriter<TOutput>, TaskContext> reducer,
	RecordReuseMode recordReuse = RecordReuseMode.Default
)
where TKey : Object, IComparable<TKey>

Parameters

input
Type: Ookii.Jumbo.Jet.Jobs.BuilderIOperationInput
The input.
reducer
Type: SystemActionTKey, IEnumerableTValue, RecordWriterTOutput, TaskContext
The reducer function.
recordReuse (Optional)
Type: Ookii.Jumbo.Jet.Jobs.BuilderRecordReuseMode
The record reuse mode.

Type Parameters

TKey
The type of the keys.
TValue
The type of the values.
TOutput
The type of the output records.

Return Value

Type: StageOperation
A StageOperation instance that can be used to further customize the operation.
Remarks
Note Note
Reduce tasks require that their input is already grouped by key. The JobBuilder class doesn't guarantee or verify this. To group the records in the same way other common MapReduce implementation do, use the SpillSortCombine(IOperationInput, Type, Type) function.
Note Note
There is no non-delegate version of this method. To use an existing map task class, simply use the Process(IOperationInput, Type) function.

This method generates a class inheriting from ReduceTaskTKey, TValue, TOutput which calls the target method of the reducer delegate from the Reduce(TKey, IEnumerableTValue, RecordWriterTOutput) method.

Note Note
The task method will be called from a completely different process than the one that is using JobBuilder, so it should not really on any external state.

If recordReuse is Default and the target method has the AllowRecordReuseAttribute attribute applied to it, that attribute will be copied to the task class. If the target method has the ProcessAllInputPartitionsAttribute attribute applied to it, that attribute will be copied to the task class.

If the target method is a public method, it will be called directly by the generated task class. Otherwise, the supplied delegate will be serialized to the task settings and used to call the method. If the target method is an instance method, the instance it belongs to will be serialized as well (this class must have the SerializableAttribute attribute).

See Also