×

Hadoop 教程

Hadoop 关于Hadoop 简介Hadoop HDFSHadoop 环境设置Hadoop 写文件Hadoop 读文件Hadoop 可靠性Hadoop 命令工具Hadoop YARNHadoop ResourceManagerHadoop NodeManagerHadoop ApplicationMasterHadoop ContainerHadoop FailoverHadoop MapReduceHadoop 读取数据Hadoop MapperHadoop ShuffleHadoop 编程Hadoop IOHadoop 测试Hadoop 安装Hadoop 配置Hadoop 监控Hadoop 参考

Hadoop 相关教程

Hadoop 大数据概述Hadoop 大数据解决方案Hadoop HDFS概述Hadoop HDFS操作Hadoop 命令参考Hadoop 流Hadoop 多节点集群

Hadoop Mapper


MapReduce - Mapper

主要是读取InputSplit的每一个Key,Value对并进行处理

public class Mapper {
    /**
     * 预处理,仅在map task启动时运行一次
     */
    protected void setup(Context context) throws  IOException, InterruptedException {
    }

    /**
     * 对于InputSplit中的每一对都会运行一次
     */
    @SuppressWarnings("unchecked")
    protected void map(KEYIN key, VALUEIN value, Context context) throws IOException, InterruptedException {
        context.write((KEYOUT) key, (VALUEOUT) value);
    }

    /**
     * 扫尾工作,比如关闭流等
     */
    protected void cleanup(Context context) throws IOException, InterruptedException {
    }

    /**
     * map task的驱动器
     */
    public void run(Context context) throws IOException, InterruptedException {
        setup(context);
        while (context.nextKeyValue()) {
            map(context.getCurrentKey(), context.getCurrentValue(), context);
        }
        cleanup(context);
    }
}

public class MapContext extends TaskInputOutputContext {
    private RecordReader reader;
    private InputSplit split;

    /**
     * Get the input split for this map.
     */
    public InputSplit getInputSplit() {
        return split;
    }

    @Override
    public KEYIN getCurrentKey() throws IOException, InterruptedException {
        return reader.getCurrentKey();
    }

    @Override
    public VALUEIN getCurrentValue() throws IOException, InterruptedException {
        return reader.getCurrentValue();
    }

    @Override
    public boolean nextKeyValue() throws IOException, InterruptedException {
        return reader.nextKeyValue();
    }
}

分类导航

关注微信下载离线手册

bootwiki移动版 bootwiki
(群号:472910771)