SmartFrog Add Hadoop support for SmartFrog Created: 14/Apr/08 04:24 PM (BST)   Updated: 08/Dec/08 11:32 AM (GMT) Component/s: _service_hadoop Affects Version/s: 3.17.004 Fix Version/s: None

Time Tracking: Issue & Sub-Tasks Issue Only Issue & Sub-Tasks Issue Only Original Estimate: 4 weeks, 4 hours Remaining Estimate: 4 weeks, 4 hours Time Spent: Not Specified Original Estimate: 4 weeks Remaining Estimate: 4 weeks Time Spent: Not Specified

Issue Links: Depends This issue depends on: SFOS-853 Need to port hadoop-default.xml to .sf SFOS-817 add AppendLines component to append lines to a text file in the filesystem, *if the lines are not already present* SFOS-836 add method to FileSystem to convert a vector of paths or FileIntf interfaces into a vector of absolute paths SFOS-1021 Bring Hadoop into the main build SFOS-855 Hadoop client-side components should be able to pick up properties from a cluster CD

Compatibility:

new feature

Sub-Tasks:

All

Open

1. Get sf-hadoop to build with needed artifacts added to our repository Resolved Steve Loughran     2. subclass Hadoop Configuration with SF support Resolved Steve Loughran     3. write component to submit jobs to a Hadoop cluster Resolved Steve Loughran     4. write component to start hadoop task tracker in-VM Resolved Steve Loughran     5. integrate Hadoop logging with SmartFrog Resolved Steve Loughran     6. Add condition component to poll cluster and job tracker status Resolved Steve Loughran     7. write component to start data node in-VM Resolved Steve Loughran     8. write component to start namenode in-VM Resolved Steve Loughran     9. write component to increase replication of existing blocks Open Steve Loughran     10. add HDFS Rebalancing component Open Steve Loughran     11. add HDFS fsck component Open Steve Loughran     12. add HDFS format component Resolved Steve Loughran 0%   13. write component to check disk space in java6+ JVM Resolved Steve Loughran     14. write components to perform filesystem create/delete/move operations, and copy data to/from the filesystem Resolved Steve Loughran     15. write component to start job tracker Resolved Steve Loughran     16. Move from direct inherited configuration to a "Cluster" CD that defines the cluster Resolved Steve Loughran     17. add component to take tuples (such as from CVS files) and write them to a hadoop file Resolved Steve Loughran 0%   18. write component to load in settings from hadoop's XML files Closed Steve Loughran     19. generate Hadoop XML config files from deployments Resolved Steve Loughran

All

Comments

Work Log

Change History

Sort Order: