API tools faq
paste
Guest User

Untitled

a guest
Oct 19th, 2018
89
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 3.41 KB | None | 0 0
raw download clone embed print report
  1. hdfs dfs -ls -R /random | grep '^d' | awk '{print $NF}'
  2.  
  3. /random/custom
  4. /random/custom/hive
  5. /random/custom/hive/jars
  6. /random/app
  7. /random/app/nifi
  8. /random/app/nifi/egispub
  9. /random/app/nifi/empgis
  10. /random/app/nifi/ods
  11. /random/app/nifi/ptcsm
  12. /random/app/nifi/tmds
  13. /random/app/nifi/ucmdb
  14. /random/app/oozie
  15. /random/app/oozie/efdm
  16. /random/app/oozie/efdm/ensco
  17. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
  18. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
  19. /random/app/oozie/efdm/mermec
  20. /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
  21. /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
  22. /random/app/oozie/gis
  23. /random/app/oozie/gis/opk
  24. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
  25. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
  26. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
  27.  
  28. /random/custom/hive/jars
  29. /random/app/nifi/egispub
  30. /random/app/nifi/empgis
  31. /random/app/nifi/ods
  32. /random/app/nifi/ptcsm
  33. /random/app/nifi/tmds
  34. /random/app/nifi/ucmdb
  35. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
  36. /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
  37. /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
  38. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
  39. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
  40.  
  41. awk -F/ 'NF <= OLDNF {print LAST}; {OLDNF = NF; LAST = $0} END {print LAST}'
  42. /random/custom/hive/jars
  43. /random/app/nifi/egispub
  44. /random/app/nifi/empgis
  45. /random/app/nifi/ods
  46. /random/app/nifi/ptcsm
  47. /random/app/nifi/tmds
  48. /random/app/nifi/ucmdb
  49. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
  50. /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
  51. /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
  52. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
  53. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
  54.  
  55. ... |sort | awk '$0 ~ pre { pre=$0; next} pre{print pre } { pre=$0 } END{ print }'
  56.  
  57. /random/custom
  58. /c/d
  59. /e/f/c
  60. /a/b
  61. /random/custom/hive/jars
  62. /random/app
  63. /random/app/nifi
  64. /random/app/nifi/egispub
  65. /random/app/nifi/empgis
  66. /random/app/nifi/ods
  67. /random/app/nifi/ptcsm
  68. /random/app/nifi/tmds
  69. /random/app/nifi/ucmdb
  70. /random/app/oozie
  71. /random/app/oozie/efdm
  72. /random/app/oozie/efdm/ensco
  73. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
  74. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
  75. /random/app/oozie/efdm/mermec
  76. /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
  77. /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
  78. /random/app/oozie/gis
  79. /random/app/oozie/gis/opk
  80. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
  81. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
  82. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
  83.  
  84. $ awk '$0 ~ pre { pre=$0; next} pre{print pre } { pre=$0 } END{ print }' <(sort infile)
  85. /a/b
  86. /c/d
  87. /e/f/c
  88. /random/app/nifi/egispub
  89. /random/app/nifi/empgis
  90. /random/app/nifi/ods
  91. /random/app/nifi/ptcsm
  92. /random/app/nifi/tmds
  93. /random/app/nifi/ucmdb
  94. /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
  95. /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
  96. /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
  97. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
  98. /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
  99. /random/custom/hive/jars
  100.  
  101. sed -E '$!N;/(.+)n1/!P;D' infile
  102.  
  103. sed '$!N;/(.{1,})n1/!P;D' infile
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!