RetroShare JSON API =================== :Cxx: C++ == How to use RetroShare JSON API Look for methods marked with +@jsonapi+ doxygen custom command into +libretroshare/src/retroshare+. The method path is composed by service instance pointer name like +rsGxsChannels+ for +RsGxsChannels+, and the method name like +createGroup+ and pass the input paramethers as a JSON object. .paramethers.json [source,json] -------------------------------------------------------------------------------- { "group":{ "mMeta":{ "mGroupName":"JSON test group", "mGroupFlags":4, "mSignFlags":520 }, "mDescription":"JSON test group description" }, "caller_data":"Here can go any kind of JSON data (even objects) that the caller want to get back together with the response" } -------------------------------------------------------------------------------- .Calling the JSON API with curl on the terminal [source,bash] -------------------------------------------------------------------------------- curl -H "Content-Type: application/json" --data @paramethers.json \ http://127.0.0.1:9092/rsGxsChannels/createGroup -------------------------------------------------------------------------------- .JSON API call result [source,json] -------------------------------------------------------------------------------- { "caller_data": "Here can go any kind of JSON data (even objects) that the caller want to get back together with the response", "retval": true, "token": 3 } -------------------------------------------------------------------------------- == Offer new RetroShare services through JSON API To offer a retroshare service through the JSON API, first of all one need find the global pointer to the service instance and document it in doxygen syntax, plus marking with the custom doxygen command +@jsonapi{RS_VERSION}+ where +RS_VERSION+ is the retroshare version in which this service became available with the current semantic (major changes to the service semantic, changes the meaning of the service itself, so the version should be updated in the documentation in that case). .Service instance pointer in rsgxschannels.h [source,cpp] -------------------------------------------------------------------------------- /** * Pointer to global instance of RsGxsChannels service implementation * @jsonapi{development} */ extern RsGxsChannels* rsGxsChannels; -------------------------------------------------------------------------------- Once the service instance itself is known to the JSON API you need to document in doxygen syntax and mark with the custom doxygen command +@jsonapi{RS_VERSION}+ the methods of the service that you want to make available through JSON API. .Offering RsGxsChannels::getChannelDownloadDirectory in rsgxschannels.h [source,cpp] -------------------------------------------------------------------------------- /** * Get download directory for the given channel * @jsonapi{development} * @param[in] channelId id of the channel * @param[out] directory reference to string where to store the path * @return false on error, true otherwise */ virtual bool getChannelDownloadDirectory( const RsGxsGroupId& channelId, std::string& directory ) = 0; -------------------------------------------------------------------------------- For each paramether you must specify if it is used as input +@param[in]+ as output +@param[out]+ or both +@param[inout]+. Paramethers and return value types must be of a type supported by +RsTypeSerializer+ which already support most basic types (+bool+, +std::string+...), +RsSerializable+ and containers of them like +std::vector+. Paramethers passed by value and by reference of those types are both supported, while passing by pointer is not supported. If your paramether or return +class+/+struct+ type is not supported yet by +RsTypeSerializer+ most convenient approach is to make it derive from +RsSerializable+ and implement +serial_process+ method like I did with +RsGxsChannelGroup+. .Deriving RsGxsChannelGroup from RsSerializable in rsgxschannels.h [source,cpp] -------------------------------------------------------------------------------- struct RsGxsChannelGroup : RsSerializable { RsGroupMetaData mMeta; std::string mDescription; RsGxsImage mImage; bool mAutoDownload; /// @see RsSerializable virtual void serial_process( RsGenericSerializer::SerializeJob j, RsGenericSerializer::SerializeContext& ctx ) { RS_SERIAL_PROCESS(mMeta); RS_SERIAL_PROCESS(mDescription); //RS_SERIAL_PROCESS(mImage); RS_SERIAL_PROCESS(mAutoDownload); } }; -------------------------------------------------------------------------------- You can do the same recursively for any member of your +struct+ that is not yet supported by +RsTypeSerializer+ like I should have done for +RsGxsImage mImage;+ but didn't have done yet because the day is so beatiful and i want to spend some time outside :D. == A bit of history === First writings about this The previous attempt of exposing a RetroShare JSON API is called +libresapi+ and unfortunatley it requires a bunch of boilerplate code when we want to expose something present in the {Cxx} API in the JSON API. As an example here you can see the libresapi that exposes part of the retroshare chat {Cxx} API and lot of boilerplate code just to convert {Cxx} objects to JSON https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L44 To avoid the {Cxx} to JSON and back conversion boilerplate code I have worked out an extension to our {Cxx} serialization code so it is capable to serialize and deserialize to JSON you can see it in this pull request https://github.com/RetroShare/RetroShare/pull/1155 So first step toward having a good API is to take advantage of the fact that RS is now capable of converting C++ objects from and to JSON. The current API is accessible via HTTP and unix socket, there is no authentication in both of them, so anyone having access to the HTTP server or to the unix socket can access the API without extra restrictions. Expecially for the HTTP API this is a big risk because also if the http server listen on 127.0.0.1 every application on the machine (even rogue javascript running on your web browser) can access that and for example on android it is not safe at all (because of that I implemented the unix socket access so at least in android API was reasonably safe) because of this. A second step to improve the API would be to implement some kind of API authentication mechanism (it would be nice that the mechanism is handled at API level and not at transport level so we can use it for any API trasport not just HTTP for example) The HTTP server used by libresapi is libmicrohttpd server that is very minimal, it doesn't provide HTTPS nor modern HTTP goodies, like server notifications, websockets etc. because the lack of support we have a token polling mechanism in libresapi to avoid polling for every thing but it is still ugly, so if we can completely get rid of polling in the API that would be really nice. I have done a crawl to look for a replacement and briefly looked at - https://www.gnu.org/software/libmicrohttpd/ - http://wolkykim.github.io/libasyncd/ - https://github.com/corvusoft/restbed - https://code.facebook.com/posts/1503205539947302/introducing-proxygen-facebook-s-c-http-framework/ - https://github.com/cmouse/yahttp taking in account a few metrics like modern HTTP goodies support, license, platform support, external dependencies and documentation it seemed to me that restbed is the more appropriate. Another source of boilerplate code into libresapi is the mapping between JSON API requests and C++ API methods as an example you can look at this https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L158 and this https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ApiServer.cpp#L253 The abstract logic of this thing is, when libreasapi get a request like +/chat/initiate_distant_chat+ then call +ChatHandler::handleInitiateDistantChatConnexion+ which in turn is just a wrapper of +RsMsgs::initiateDistantChatConnexion+ all this process is basically implemented as boilerplate code and would be unnecessary in a smarter design of the API because almost all the information needed is already present in the C++ API +libretroshare/src/retroshare+. So a third step to improve the JSON API would be to remove this source of boilerplate code by automatizing the mapping between C++ and JSON API call. This may result a little tricky as language parsing or other adevanced things may be required. Hope this dive is useful for you + Cheers + G10h4ck === Second writings about this I have been investigating a bit more about: [verse, G10h4ck] ________________________________________________________________________________ So a third step to improve the JSON API would be to remove this source of boilerplate code by automatizing the mapping between C++ and JSON API call ________________________________________________________________________________ After spending some hours investigating this topic the most reasonable approach seems to: 1. Properly document headers in +libretroshare/src/retroshare/+ in doxygen syntax specifying wihich params are input and/or output (doxygen sysntax for this is +@param[in/out/inout]+) this will be the API documentation too. 2. At compile time use doxygen to generate XML description of the headers and use the XML to generate the JSON api server stub. http://www.stack.nl/~dimitri/doxygen/manual/customize.html#xmlgenerator 3. Enjoy