API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Sep 16th, 2015
112
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 12.00 KB | None | 0 0
raw download clone embed print report
  1. Assignment
  2.  
  3. The goal of the assignment is to build a user profile management service. It will have methods to create, change and get the user profile.
  4. The service uses Redis as main database to keep latest data and hadoop to store all activity. External databases will be mocked so development would need less time for setup.
  5.  
  6.  
  7.  
  8. NB! The solution must not be publicly available in code repositories like GitHub etc before June 15, 2014.
  9.  
  10.  
  11.  
  12. The main points of implementation:
  13.  
  14. · External API and logic behind it. API is descibed below. Also the data model.
  15.  
  16. · Idempotency and change conflicts. Explained below.
  17.  
  18. · Background process to push events to a separate database and delete them from redis.
  19.  
  20. · Internal event queue/cache.
  21.  
  22.  
  23.  
  24.  
  25.  
  26. Restrictions and Help
  27.  
  28. You will be provided a sample project, that has an example service „hello world“ implemented. In this example also redis is mocked and interface is provided to communicate with hadoop datastore.
  29.  
  30. Usage of external libraries is restricted. Only existing libraries/dependencies can be used. Everything else should be implemented in the project.
  31.  
  32. Implement as much as you can during the given timeframe.
  33.  
  34.  
  35.  
  36. API description
  37.  
  38. · /user/create
  39.  
  40. o Request parameters:
  41.  
  42. § actor=<string>, mandatory
  43.  
  44. § username=<string>, mandatory
  45.  
  46. § changeid=<string>, mandatory
  47.  
  48. § firstname=<string>
  49.  
  50. § lastname=<string>
  51.  
  52. § birthdate=<YYYY.MM.DD>
  53.  
  54. § currency=<string>
  55.  
  56. § language=<string>
  57.  
  58. § email=<string>, mandatory
  59.  
  60. o Response parameters:
  61.  
  62. § userid=<string>, mandatory on success
  63.  
  64. § errorText=<string>, optional
  65.  
  66. o Response errors:
  67.  
  68. § 200 – success (or retry)
  69.  
  70. § 400 – invalid input
  71.  
  72. § 409 – Duplicate username
  73.  
  74. § 500 – system error
  75.  
  76. · /user/change
  77.  
  78. o Request parameters:
  79.  
  80. § actor=<string>, mandatory
  81.  
  82. § userid=<string>, mandatory
  83.  
  84. § version=<integer>, mandatory – what version of user profile was used to make the decision of changing it.
  85.  
  86. § changeid=<string>, mandatory
  87.  
  88. § username=<string>
  89.  
  90. § firstname=<string>
  91.  
  92. § lastname=<string>
  93.  
  94. § birthdate=<YYYY.MM.DD>
  95.  
  96. § currency=<string>
  97.  
  98. § language=<string>
  99.  
  100. § email=<string>
  101.  
  102. o Response parameters:
  103.  
  104. § errorText=<string>, optional
  105.  
  106. § version=<integer>, current version
  107.  
  108. o Response errors
  109.  
  110. § 200 – success (or retry)
  111.  
  112. § 400 – invalid input
  113.  
  114. § 404 – user profile not found
  115.  
  116. § 409 – change conflict or duplicate username
  117.  
  118. § 500 – system error
  119.  
  120. · /user/get
  121.  
  122. o Request parameters:
  123.  
  124. § actor=<string>, mandatory
  125.  
  126. § userid=<string>, username or userid mandatory
  127.  
  128. § username=<string>, username or userid mandatory
  129.  
  130. o Response parameters
  131.  
  132. § errorText=<string>, optional
  133.  
  134. § userid=<string>, mandatory on success
  135.  
  136. § username=<string>, mandatory on success
  137.  
  138. § firstname=<string>
  139.  
  140. § lastname=<string>
  141.  
  142. § birthdate=<YYYY.MM.DD>
  143.  
  144. § currency=<string>
  145.  
  146. § language=<string>
  147.  
  148. § email=<string>, mandatory on success
  149.  
  150. § signuptime=<YYYY.MM.DD HH:mm:ss.SSS>, mandatory on success
  151.  
  152. § version=<integer>, mandatory on success
  153.  
  154. § changetime=<YYYY.MM.DD HH:mm:ss.SSS>, mandatory on success
  155.  
  156. o Response errors
  157.  
  158. § 200 – success
  159.  
  160. § 400 – invalid input
  161.  
  162. § 404 – user profile not found
  163.  
  164. § 500 – system error
  165.  
  166.  
  167.  
  168. Data structures
  169.  
  170. · User profile object – Contains latest user profile info.
  171.  
  172. o TTL – indefinite
  173.  
  174. o Key – USER:PROFILE:<userid>
  175.  
  176. o Data (in json format)
  177.  
  178. § userid
  179.  
  180. § username
  181.  
  182. § firstname
  183.  
  184. § lastname
  185.  
  186. § birthdate
  187.  
  188. § currency
  189.  
  190. § language
  191.  
  192. § email
  193.  
  194. § signuptime
  195.  
  196. § version
  197.  
  198. § changetime
  199.  
  200. · User profile change versions – Used during change process to avoid data races.
  201.  
  202. o TTL – 1d
  203.  
  204. o Key – USER:PVERSIONS:<userid>:<version>
  205.  
  206. o Data (in json format)
  207.  
  208. § Same as User object.
  209.  
  210. · Profile change changeids – Used during create and change to do idempotency checks.
  211.  
  212. o TTL – 1d
  213.  
  214. o Key – USER:PCHANGEID:<userid>:<changeid>
  215.  
  216. o Data (in json format)
  217.  
  218. § version
  219.  
  220. · Usernames – Mapping of usernames to userids. Needed to make initial resolution, when username is used for reference.
  221.  
  222. o TTL – indefinite
  223.  
  224. o Key – USER:USERNAMES:username
  225.  
  226. o Data (in json format)
  227.  
  228. § userid
  229.  
  230. · Events – temporary event history backup, before sending to hadoop or some other storage.
  231.  
  232. o TTL – indefinite
  233.  
  234. o Key – EVENTS:eventid
  235.  
  236. o Data (in json format)
  237.  
  238. § id – event id.
  239.  
  240. § name – create, change or get. Event name.
  241.  
  242. § userid – reference to user profile
  243.  
  244. § actor – reference to who is causing the event
  245.  
  246. § ip – address of the request
  247.  
  248. § timestamp – time of the event in milliseconds
  249.  
  250. § request – json string of request
  251.  
  252.  
  253.  
  254. Concepts explained
  255.  
  256. · Itempotency – The internet is not bulletproof. There will be networking issues temporarily somewhere. Good system has a way to deal with retrys.
  257.  
  258. · Entity versioning – Different parts of the system might show or process the same entity (admin, user facing client, other services). These parts of the system rarely change the same parameters, but do change at the same time. Versioning allows to identify conflicts or get around it, if different parts of the system change different parameters at the same time.
  259.  
  260. o Example:
  261.  
  262. § User opens its profile page. Profile data version 3 is shown.
  263.  
  264. § User is slow. It does not make an action within 5min.
  265.  
  266. § Meanwhile a system process changes user segmentation group. Version is updated to 4.
  267.  
  268. § Now user does the update (firstname typo). Version=3 is sent in request.
  269.  
  270. § System can compare version 3 and new change request. It identifies, that firstname is changed. Firstname was not changed with version 4 update, so there are no conflicts.
  271.  
  272. · Optimistic locking – Redis does not support transactions as such. Instead optimistic locking is used.
  273.  
  274. o How to use: http://redis.io/topics/transactions
  275.  
  276. o Example:
  277.  
  278. § WATCH key – start optimistic lock
  279.  
  280. § GET key – get value
  281.  
  282. § Processing in server...
  283.  
  284. § MULTI – redis transaction (means just, that commands are executed in one batch)
  285.  
  286. § SET key ... – change data
  287.  
  288. § EXEC – end transaction (actual execution of commands)
  289.  
  290. o Example java:
  291.  
  292. IRedisClient jedis = getRedisManager().getJedis();
  293.  
  294. try {
  295.  
  296. // start transaction and get profile
  297.  
  298. jedis.watch(profileKey);
  299.  
  300. String profilejson = jedis.get(key);
  301.  
  302. ...
  303.  
  304. // start batch
  305.  
  306. IRedisClient t = jedis.multi();
  307.  
  308. t.set(profileKey, newprofilejson);
  309.  
  310. t.exec();
  311.  
  312. } finally {
  313.  
  314. getRedisManager().returnJedis(jedis);
  315.  
  316. }
  317.  
  318.  
  319.  
  320.  
  321.  
  322. Examples
  323.  
  324. Redis communication (in HelloWorldService):
  325.  
  326. IRedisClient jedis = RedisManager.getInstance().getJedis();
  327.  
  328. try {
  329.  
  330. jedis.set(key, value);
  331.  
  332. } catch (Exception e) {
  333.  
  334. throw new RuntimeException("Redis issue!", e);
  335.  
  336. } finally {
  337.  
  338. RedisManager.getInstance().returnJedis(jedis);
  339.  
  340. }
  341.  
  342.  
  343.  
  344. POJO to json conversion (in HelloWorldService):
  345.  
  346. // pojo to json
  347.  
  348. String json = mapper.writeValueAsString(result);
  349.  
  350. System.out.println( "to json> " + json.getClass() + "::" + json );
  351.  
  352.  
  353.  
  354. // json to pojo
  355.  
  356. HelloWorldBuilder builder = mapper.readValue(json, HelloWorldBuilder.class);
  357.  
  358. System.out.println( "from json> " + builder.build() );
  359.  
  360.  
  361.  
  362.  
  363.  
  364. API usage scenarios:
  365.  
  366. Create user missing parameters
  367.  
  368. http://localhost:9090/user/create?actor=me&email=someone@meil.me&changeid=34j53h4
  369.  
  370. {"errortext":"Invalid input: Parameter username is mandatory."}
  371.  
  372. Create user
  373.  
  374. http://localhost:9090/user/create?actor=me&email=someone@meil.me&changeid=3243243kj4dsfjdsvf32kj&username=someone1
  375.  
  376. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b"}
  377.  
  378. Get user
  379.  
  380. http://localhost:9090/user/get?actor=me&username=someone1
  381.  
  382. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"2","username":"someone1","firstname":"newname",
  383.  
  384. "lastname":null,"birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  385.  
  386. "changetime":"2015.04.110 14:55:36.853","signupdate":"2015.04.110 14:55:03.090"}
  387.  
  388. Change user firstname
  389.  
  390. http://localhost:9090/user/change?actor=me&username=someone1&changeid=1000&firstname=newname&version=1
  391.  
  392. {"version":"2"}
  393.  
  394. Get user
  395.  
  396. http://localhost:9090/user/get?actor=me&username=someone1
  397.  
  398. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"2","username":"someone1","firstname":"newname",
  399.  
  400. "lastname":null,"birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  401.  
  402. "changetime":"2015.04.110 14:55:36.853","signupdate":"2015.04.110 14:55:03.090"}
  403.  
  404. Create retry
  405.  
  406. http://localhost:9090/user/create?actor=me&email=someone@meil.me&changeid=3243243kj4dsfjdsvf32kj&username=someone1
  407.  
  408. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b "}
  409.  
  410. Create conflict
  411.  
  412. http://localhost:9090/user/create?actor=me&email=someone@meil.me&changeid=k45n3jk4b5&username=someone1
  413.  
  414. {"errortext":"Username already in use."}
  415.  
  416. Get user
  417.  
  418. http://localhost:9090/user/get?actor=me&username=someone1
  419.  
  420. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"2","username":"someone1","firstname":"newname",
  421.  
  422. "lastname":null,"birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  423.  
  424. "changetime":"2015.04.110 14:55:36.853","signupdate":"2015.04.110 14:55:03.090"}
  425.  
  426. Invalid create retry
  427.  
  428. http://localhost:9090/user/create?actor=me&email=someone@meil.me&changeid=3243243kj4dsfjdsvf32kj&username=someone1&lastname=tamm
  429.  
  430. {"errortext":"lastname retry conflict. Was null, now tamm."}
  431.  
  432. Non-conflicting change
  433.  
  434. http://localhost:9090/user/change?actor=me&username=someone1&changeid=1001&lastname=newname&version=1
  435.  
  436. {"version":"3"}
  437.  
  438. Get user
  439.  
  440. http://localhost:9090/user/get?actor=me&username=someone1
  441.  
  442. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"3","username":"someone1","firstname":"newname",
  443.  
  444. "lastname":"newname","birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  445.  
  446. "changetime":"2015.04.110 15:10:11.417","signupdate":"2015.04.110 15:09:59.308"}
  447.  
  448. Conflicting change
  449.  
  450. http://localhost:9090/user/change?actor=me&username=someone1&changeid=1002&firstname=tester&version=1
  451.  
  452. {"errortext":"Firstname change conflict."}
  453.  
  454. Get user
  455.  
  456. http://localhost:9090/user/get?actor=me&username=someone1
  457.  
  458. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"3","username":"someone1","firstname":"newname",
  459.  
  460. "lastname":"newname","birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  461.  
  462. "changetime":"2015.04.110 15:10:11.417","signupdate":"2015.04.110 15:09:59.308"}
  463.  
  464. Change retry
  465.  
  466. http://localhost:9090/user/change?actor=me&username=someone1&changeid=1001&lastname=newname&version=1
  467.  
  468. {"version":"3"}
  469.  
  470. User does not exist
  471.  
  472. http://localhost:9090/user/get?actor=me&username=someone2
  473.  
  474. {"errortext":"Profile not found (username): someone2"}
  475.  
  476. User does not exist
  477.  
  478. http://localhost:9090/user/get?actor=me&userid=someone2
  479.  
  480. {"errortext":"Profile not found (userid): someone2"}
  481.  
  482. Change username
  483.  
  484. http://localhost:9090/user/change?actor=me&userid=8b8963a5-7076-4ba2-acbe-156675c0b93b&changeid=1004&username=tester1&version=3
  485.  
  486. {"version":"4"}
  487.  
  488. Get user
  489.  
  490. http://localhost:9090/user/get?actor=me&username=someone1
  491.  
  492. {"errortext":"User profile not found by username: someone1"}
  493.  
  494. Get user
  495.  
  496. http://localhost:9090/user/get?actor=me&username=tester1
  497.  
  498. {"userid":"8b8963a5-7076-4ba2-acbe-156675c0b93b","version":"4","username":"tester1","firstname":"newname",
  499.  
  500. "lastname":"newname","birthdate":null,"currency":null,"language":null,"email":"someone@meil.me",
  501.  
  502. "changetime":"2015.04.110 15:23:37.510","signupdate":"2015.04.110 15:23:22.072"}
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!