Implement automatic JSON API generation

qmake file add jsonapi-generator target to compile JSON API generator
qmake files add rs_jsonapi CONFIG option to enable/disable JSON API at compile
  time
RsTypeSerializer pass down same serialization flags when creating new context
  for nested objects serial job
RsGxsChannels expose a few methods through JSON API as example
Derive a few GXS types (RsGxsChannelGroup, RsGxsChannelPost, RsGxsFile,
  RsMsgMetaData) from RsSerializables so they can be used for the JSON API
Create RsGenericSerializer::SERIALIZATION_FLAG_YIELDING so JSON objects that
  miss some fields can be still deserialized, this improve API usability
SerializeContext offer friendly constructor with default paramethers
Add restbed 4.6 library as git submodule as most systems doesn't have it yet
Add a bit of documentation about JSON API into jsonapi-generator/README.adoc
Add JsonApiServer class to expose the JSON API via HTTP protocol

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "supportlibs/restbed"]
+	path = supportlibs/restbed
+	url = https://github.com/Corvusoft/restbed.git

RetroShare.pro

@@ -10,7 +10,13 @@ retrotor {
 } else {
 	SUBDIRS += libbitdht
 	libbitdht.file = libbitdht/src/libbitdht.pro
-	libretroshare.depends = openpgpsdk libbitdht
+    libretroshare.depends = openpgpsdk libbitdht
+}
+
+rs_jsonapi {
+    SUBDIRS += jsonapi-generator
+    jsonapi-generator.file = jsonapi-generator/src/jsonapi-generator.pro
+    libretroshare.depends += jsonapi-generator
 }
 
 SUBDIRS += libretroshare

jsonapi-generator/README.adoc

@@ -0,0 +1,227 @@
+RetroShare JSON API
+===================
+
+:Cxx: C++
+
+== How to use RetroShare JSON API
+
+Look for methods marked with +@jsonapi+ doxygen custom command into
++libretroshare/src/retroshare+. The method path is composed by service instance
+pointer name like +rsGxsChannels+ for +RsGxsChannels+, and the method name like
++createGroup+ and pass the input paramethers as a JSON object.
+
+.paramethers.json
+[source,json]
+--------------------------------------------------------------------------------
+{
+    "group":{
+        "mMeta":{
+            "mGroupName":"JSON test group",
+            "mGroupFlags":4,
+            "mSignFlags":520
+        },
+        "mDescription":"JSON test group description"
+    },
+    "caller_data":"Here can go any kind of JSON data (even objects) that the caller want to get back together with the response"
+}
+--------------------------------------------------------------------------------
+
+.Calling the JSON API with curl on the terminal
+[source,bash]
+--------------------------------------------------------------------------------
+curl -H "Content-Type: application/json" --data @paramethers.json \
+	http://127.0.0.1:9092/rsGxsChannels/createGroup
+--------------------------------------------------------------------------------
+
+.JSON API call result
+[source,json]
+--------------------------------------------------------------------------------
+{
+    "caller_data": "Here can go any kind of JSON data (even objects) that the caller want to get back together with the response",
+    "retval": true,
+    "token": 3
+}
+--------------------------------------------------------------------------------
+
+
+== Offer new RetroShare services through JSON API
+
+To offer a retroshare service through the JSON API, first of all one need find
+the global pointer to the service instance and document it in doxygen syntax,
+plus marking with the custom doxygen command +@jsonapi{RS_VERSION}+ where
++RS_VERSION+ is the retroshare version in which this service became available
+with the current semantic (major changes to the service semantic, changes the
+meaning of the service itself, so the version should be updated in the
+documentation in that case).
+
+.Service instance pointer in rsgxschannels.h
+[source,cpp]
+--------------------------------------------------------------------------------
+/**
+ * Pointer to global instance of RsGxsChannels service implementation
+ * @jsonapi{development}
+ */
+extern RsGxsChannels* rsGxsChannels;
+--------------------------------------------------------------------------------
+
+
+Once the service instance itself is known to the JSON API you need to document
+in doxygen syntax and mark with the custom doxygen command
++@jsonapi{RS_VERSION}+ the methods of the service that you want to make
+available through JSON API.
+
+.Offering RsGxsChannels::getChannelDownloadDirectory in rsgxschannels.h
+[source,cpp]
+--------------------------------------------------------------------------------
+	/**
+	 * Get download directory for the given channel
+	 * @jsonapi{development}
+	 * @param[in] channelId id of the channel
+	 * @param[out] directory reference to string where to store the path
+	 * @return false on error, true otherwise
+	 */
+	virtual bool getChannelDownloadDirectory( const RsGxsGroupId& channelId,
+	                                          std::string& directory ) = 0;
+--------------------------------------------------------------------------------
+
+For each paramether you must specify if it is used as input +@param[in]+ as
+output +@param[out]+ or both +@param[inout]+. Paramethers and return value
+types must be of a type supported by +RsTypeSerializer+ which already support
+most basic types (+bool+, +std::string+...), +RsSerializable+ and containers of
+them like +std::vector<std::string>+. Paramethers passed by value and by
+reference of those types are both supported, while passing by pointer is not
+supported. If your paramether or return +class+/+struct+ type is not supported
+yet by +RsTypeSerializer+ most convenient approach is to make it derive from
++RsSerializable+ and implement +serial_process+ method like I did with
++RsGxsChannelGroup+.
+
+.Deriving RsGxsChannelGroup from RsSerializable in rsgxschannels.h
+[source,cpp]
+--------------------------------------------------------------------------------
+struct RsGxsChannelGroup : RsSerializable
+{
+	RsGroupMetaData mMeta;
+	std::string mDescription;
+	RsGxsImage mImage;
+
+	bool mAutoDownload;
+
+	/// @see RsSerializable
+	virtual void serial_process( RsGenericSerializer::SerializeJob j,
+	                             RsGenericSerializer::SerializeContext& ctx )
+	{
+		RS_SERIAL_PROCESS(mMeta);
+		RS_SERIAL_PROCESS(mDescription);
+		//RS_SERIAL_PROCESS(mImage);
+		RS_SERIAL_PROCESS(mAutoDownload);
+	}
+};
+--------------------------------------------------------------------------------
+
+You can do the same recursively for any member of your +struct+ that is not yet
+supported by +RsTypeSerializer+ like I should have done for +RsGxsImage mImage;+
+but didn't have done yet because the day is so beatiful and i want to spend some
+time outside :D.
+
+
+== A bit of history
+
+=== First writings about this
+
+The previous attempt of exposing a RetroShare JSON API is called +libresapi+ and
+unfortunatley it requires a bunch of boilerplate code when we want to expose
+something present in the {Cxx} API in the JSON API.
+
+As an example here you can see the libresapi that exposes part of the retroshare
+chat {Cxx} API and lot of boilerplate code just to convert {Cxx} objects to JSON
+
+https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L44
+
+To avoid the {Cxx} to JSON and back conversion boilerplate code I have worked out
+an extension to our {Cxx} serialization code so it is capable to serialize and
+deserialize to JSON you can see it in this pull request
+
+https://github.com/RetroShare/RetroShare/pull/1155
+
+So first step toward having a good API is to take advantage of the fact that RS
+is now capable of converting C++ objects from and to JSON.
+
+The current API is accessible via HTTP and unix socket, there is no
+authentication in both of them, so anyone having access to the HTTP server or to
+the unix socket can access the API without extra restrictions.
+Expecially for the HTTP API this is a big risk because also if the http server
+listen on 127.0.0.1 every application on the machine (even rogue javascript
+running on your web browser) can access that and for example on android it is
+not safe at all (because of that I implemented the unix socket access so at
+least in android API was reasonably safe) because of this.
+
+A second step to improve the API would be to implement some kind of API
+authentication mechanism (it would be nice that the mechanism is handled at API
+level and not at transport level so we can use it for any API trasport not just
+HTTP for example)
+
+The HTTP server used by libresapi is libmicrohttpd server that is very minimal,
+it doesn't provide HTTPS nor modern HTTP goodies, like server notifications,
+websockets etc. because the lack of support we have a token polling mechanism in
+libresapi to avoid polling for every thing but it is still ugly, so if we can
+completely get rid of polling in the API that would be really nice.
+I have done a crawl to look for a replacement and briefly looked at
+
+- https://www.gnu.org/software/libmicrohttpd/
+- http://wolkykim.github.io/libasyncd/
+- https://github.com/corvusoft/restbed
+- https://code.facebook.com/posts/1503205539947302/introducing-proxygen-facebook-s-c-http-framework/
+- https://github.com/cmouse/yahttp
+
+taking in account a few metrics like modern HTTP goodies support, license,
+platform support, external dependencies and documentation it seemed to me that
+restbed is the more appropriate.
+
+Another source of boilerplate code into libresapi is the mapping between JSON
+API requests and C++ API methods as an example you can look at this
+
+https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L158
+
+and this
+
+https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ApiServer.cpp#L253
+
+The abstract logic of this thing is, when libreasapi get a request like
++/chat/initiate_distant_chat+ then call
++ChatHandler::handleInitiateDistantChatConnexion+ which in turn is just a
+wrapper of +RsMsgs::initiateDistantChatConnexion+ all this process is basically
+implemented as boilerplate code and would be unnecessary in a smarter design of
+the API because almost all the information needed is already present in the
+C++ API +libretroshare/src/retroshare+.
+
+So a third step to improve the JSON API would be to remove this source of
+boilerplate code by automatizing the mapping between C++ and JSON API call.
+
+This may result a little tricky as language parsing or other adevanced things
+may be required.
+
+Hope this dive is useful for you +
+Cheers +
+G10h4ck
+
+=== Second writings about this
+
+I have been investigating a bit more about:
+[verse, G10h4ck]
+________________________________________________________________________________
+So a third step to improve the JSON API would be to remove this source of
+boilerplate code by automatizing the mapping between C++ and JSON API call
+________________________________________________________________________________
+
+After spending some hours investigating this topic the most reasonable approach
+seems to:
+
+1. Properly document headers in +libretroshare/src/retroshare/+ in doxygen syntax
+specifying wihich params are input and/or output (doxygen sysntax for this is
++@param[in/out/inout]+) this will be the API documentation too.
+
+2. At compile time use doxygen to generate XML description of the headers and use
+the XML to generate the JSON api server stub.
+http://www.stack.nl/~dimitri/doxygen/manual/customize.html#xmlgenerator
+
+3. Enjoy

jsonapi-generator/src/jsonapi-generator-doxygen.conf

@@ -0,0 +1,230 @@
+DOXYFILE_ENCODING      = UTF-8
+PROJECT_NAME           = "libretroshare"
+#OUTPUT_DIRECTORY       =
+
+ALIASES += jsonapi{1}="\xmlonly<jsonapi minversion=\"\1\"/>\endxmlonly"
+
+# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the
+# documentation from any documented member that it re-implements.
+# The default value is: YES.
+
+INHERIT_DOCS           = YES
+
+# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments
+# according to the Markdown format, which allows for more readable
+# documentation. See http://daringfireball.net/projects/markdown/ for details.
+# The output of markdown processing is further processed by doxygen, so you can
+# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in
+# case of backward compatibilities issues.
+# The default value is: YES.
+
+MARKDOWN_SUPPORT       = YES
+
+# When enabled doxygen tries to link words that correspond to documented
+# classes, or namespaces to their corresponding documentation. Such a link can
+# be prevented in individual cases by putting a % sign in front of the word or
+# globally by setting AUTOLINK_SUPPORT to NO.
+# The default value is: YES.
+
+AUTOLINK_SUPPORT       = YES
+
+
+# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in
+# documentation are documented, even if no documentation was available. Private
+# class members and static file members will be hidden unless the
+# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES.
+# Note: This will also disable the warnings about undocumented members that are
+# normally produced when WARNINGS is set to YES.
+# The default value is: NO.
+
+EXTRACT_ALL            = YES
+
+
+# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all
+# undocumented members inside documented classes or files. If set to NO these
+# members will be included in the various overviews, but no documentation
+# section is generated. This option has no effect if EXTRACT_ALL is enabled.
+# The default value is: NO.
+
+HIDE_UNDOC_MEMBERS     = NO
+
+# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all
+# undocumented classes that are normally visible in the class hierarchy. If set
+# to NO, these classes will be included in the various overviews. This option
+# has no effect if EXTRACT_ALL is enabled.
+# The default value is: NO.
+
+HIDE_UNDOC_CLASSES     = NO
+
+
+#---------------------------------------------------------------------------
+# Configuration options related to the input files
+#---------------------------------------------------------------------------
+
+# The INPUT tag is used to specify the files and/or directories that contain
+# documented source files. You may enter file names like myfile.cpp or
+# directories like /usr/src/myproject. Separate the files or directories with
+# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
+# Note: If this tag is empty the current directory is searched.
+
+#INPUT                  =
+
+# This tag can be used to specify the character encoding of the source files
+# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
+# libiconv (or the iconv built into libc) for the transcoding. See the libiconv
+# documentation (see: https://www.gnu.org/software/libiconv/) for the list of
+# possible encodings.
+# The default value is: UTF-8.
+
+INPUT_ENCODING         = UTF-8
+
+# If the value of the INPUT tag contains directories, you can use the
+# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and
+# *.h) to filter out the source-files in the directories.
+#
+# Note that for custom extensions or not directly supported extensions you also
+# need to set EXTENSION_MAPPING for the extension otherwise the files are not
+# read by doxygen.
+#
+# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp,
+# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h,
+# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc,
+# *.m, *.markdown, *.md, *.mm, *.dox, *.py, *.pyw, *.f90, *.f95, *.f03, *.f08,
+# *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf and *.qsf.
+
+FILE_PATTERNS          = *.c \
+                         *.cc \
+                         *.cxx \
+                         *.cpp \
+                         *.c++ \
+                         *.java \
+                         *.ii \
+                         *.ixx \
+                         *.ipp \
+                         *.i++ \
+                         *.inl \
+                         *.idl \
+                         *.ddl \
+                         *.odl \
+                         *.h \
+                         *.hh \
+                         *.hxx \
+                         *.hpp \
+                         *.h++ \
+                         *.cs \
+                         *.d \
+                         *.php \
+                         *.php4 \
+                         *.php5 \
+                         *.phtml \
+                         *.inc \
+                         *.m \
+                         *.markdown \
+                         *.md \
+                         *.mm \
+                         *.dox \
+                         *.py \
+                         *.pyw \
+                         *.f90 \
+                         *.f95 \
+                         *.f03 \
+                         *.f08 \
+                         *.f \
+                         *.for \
+                         *.tcl \
+                         *.vhd \
+                         *.vhdl \
+                         *.ucf \
+                         *.qsf
+
+# The RECURSIVE tag can be used to specify whether or not subdirectories should
+# be searched for input files as well.
+# The default value is: NO.
+
+RECURSIVE              = YES
+
+# The EXCLUDE tag can be used to specify files and/or directories that should be
+# excluded from the INPUT source files. This way you can easily exclude a
+# subdirectory from a directory tree whose root is specified with the INPUT tag.
+#
+# Note that relative paths are relative to the directory from which doxygen is
+# run.
+
+EXCLUDE                =
+
+# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or
+# directories that are symbolic links (a Unix file system feature) are excluded
+# from the input.
+# The default value is: NO.
+
+EXCLUDE_SYMLINKS       = NO
+
+# If the value of the INPUT tag contains directories, you can use the
+# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude
+# certain files from those directories.
+#
+# Note that the wildcards are matched against the file with absolute path, so to
+# exclude all test directories for example use the pattern */test/*
+
+EXCLUDE_PATTERNS       =
+
+# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names
+# (namespaces, classes, functions, etc.) that should be excluded from the
+# output. The symbol name can be a fully qualified name, a word, or if the
+# wildcard * is used, a substring. Examples: ANamespace, AClass,
+# AClass::ANamespace, ANamespace::*Test
+#
+# Note that the wildcards are matched against the file with absolute path, so to
+# exclude all test directories use the pattern */test/*
+
+EXCLUDE_SYMBOLS        =
+
+
+#---------------------------------------------------------------------------
+# Configuration options related to the HTML output
+#---------------------------------------------------------------------------
+
+# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output
+# The default value is: YES.
+
+GENERATE_HTML          = NO
+
+# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
+# The default value is: YES.
+
+GENERATE_LATEX         = NO
+
+# If the GENERATE_XML tag is set to YES, doxygen will generate an XML file that
+# captures the structure of the code including all documentation.
+# The default value is: NO.
+
+GENERATE_XML           = YES
+
+
+#---------------------------------------------------------------------------
+# Configuration options related to the preprocessor
+#---------------------------------------------------------------------------
+
+# If the ENABLE_PREPROCESSING tag is set to YES, doxygen will evaluate all
+# C-preprocessor directives found in the sources and include files.
+# The default value is: YES.
+
+ENABLE_PREPROCESSING   = YES
+
+# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names
+# in the source code. If set to NO, only conditional compilation will be
+# performed. Macro expansion can be done in a controlled way by setting
+# EXPAND_ONLY_PREDEF to YES.
+# The default value is: NO.
+# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
+
+MACRO_EXPANSION        = NO
+
+# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
+# the macro expansion is limited to the macros specified with the PREDEFINED and
+# EXPAND_AS_DEFINED tags.
+# The default value is: NO.
+# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
+
+EXPAND_ONLY_PREDEF     = NO
+

jsonapi-generator/src/jsonapi-generator.cpp

@@ -0,0 +1,239 @@
+/*
+ * RetroShare JSON API
+ * Copyright (C) 2018  Gioacchino Mazzurco <gio@eigenlab.org>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <QCoreApplication>
+#include <QDebug>
+#include <QtXml>
+#include <QDirIterator>
+#include <QFileInfo>
+#include <iterator>
+
+struct MethodParam
+{
+	MethodParam() : in(false), out(false) {}
+
+	QString type;
+	QString name;
+	bool in;
+	bool out;
+};
+
+int main(int argc, char *argv[])
+{
+	if(argc != 3)
+		qFatal("Usage: jsonapi-generator SOURCE_PATH OUTPUT_PATH");
+
+	QString sourcePath(argv[1]);
+	QString outputPath(argv[2]);
+	QString doxPrefix(outputPath+"/xml/");
+
+	QString wrappersDefFilePath(outputPath + "/jsonapi-wrappers.cpp");
+	QFile wrappersDefFile(wrappersDefFilePath);
+	wrappersDefFile.remove();
+	if(!wrappersDefFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
+		qFatal(QString("Can't open: " + wrappersDefFilePath).toLatin1().data());
+
+	QString wrappersDeclFilePath(outputPath + "/jsonapi-wrappers.h");
+	QFile wrappersDeclFile(wrappersDeclFilePath);
+	wrappersDeclFile.remove();
+	if(!wrappersDeclFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
+		qFatal(QString("Can't open: " + wrappersDeclFilePath).toLatin1().data());
+
+	QString wrappersRegisterFilePath(outputPath + "/jsonapi-register.inl");
+	QFile wrappersRegisterFile(wrappersRegisterFilePath);
+	wrappersRegisterFile.remove();
+	if(!wrappersRegisterFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
+		qFatal(QString("Can't open: " + wrappersRegisterFilePath).toLatin1().data());
+
+
+	QDirIterator it(doxPrefix, QStringList() << "*8h.xml", QDir::Files);
+	while(it.hasNext())
+	{
+		QDomDocument hDoc;
+		QString hFilePath(it.next());
+		QString parseError; int line, column;
+		QFile hFile(hFilePath);
+		if (!hFile.open(QIODevice::ReadOnly) ||
+		        !hDoc.setContent(&hFile, &parseError, &line, &column))
+		{
+			qWarning() << "Error parsing:" << hFilePath
+			           << parseError << line << column;
+			continue;
+		}
+
+		QFileInfo hfi(hFile);
+		QString headerFileName(hfi.fileName());
+		headerFileName.replace(QString("_8h.xml"), QString(".h"));
+
+		QDomNodeList sectiondefs = hDoc.elementsByTagName("sectiondef");
+		for(int j = 0; j < sectiondefs.size(); ++j)
+		{
+			QDomElement sectDef = sectiondefs.item(j).toElement();
+
+			if( sectDef.attributes().namedItem("kind").nodeValue() != "var"
+			        || sectDef.elementsByTagName("jsonapi").isEmpty() )
+				continue;
+
+			QString instanceName =
+			        sectDef.elementsByTagName("name").item(0).toElement().text();
+			QString typeName =
+			        sectDef.elementsByTagName("ref").item(0).toElement().text();
+			QString typeFilePath(doxPrefix);
+			typeFilePath += sectDef.elementsByTagName("ref").item(0)
+			        .attributes().namedItem("refid").nodeValue();
+			typeFilePath += ".xml";
+
+			QDomDocument typeDoc;
+			QFile typeFile(typeFilePath);
+			if ( !typeFile.open(QIODevice::ReadOnly) ||
+			     !typeDoc.setContent(&typeFile, &parseError, &line, &column) )
+			{
+				qWarning() << "Error parsing:" << typeFilePath
+				           << parseError << line << column;
+				continue;
+			}
+
+			QDomNodeList members = typeDoc.elementsByTagName("member");
+			for (int i = 0; i < members.size(); ++i)
+			{
+				QDomNode member = members.item(i);
+				QString refid(member.attributes().namedItem("refid").nodeValue());
+				QString methodName(member.firstChildElement("name").toElement().text());
+				QString wrapperName(instanceName+methodName+"Wrapper");
+
+				QDomDocument defDoc;
+				QString defFilePath(doxPrefix + refid.split('_')[0] + ".xml");
+				QFile defFile(defFilePath);
+				if ( !defFile.open(QIODevice::ReadOnly) ||
+				     !defDoc.setContent(&defFile, &parseError, &line, &column) )
+				{
+					qWarning() << "Error parsing:" << defFilePath
+					           << parseError << line << column;
+					continue;
+				}
+
+				QDomElement memberdef;
+				QDomNodeList memberdefs = typeDoc.elementsByTagName("memberdef");
+				for (int k = 0; k < memberdefs.size(); ++k)
+				{
+					QDomElement tmpMBD = memberdefs.item(k).toElement();
+					QString tmpId = tmpMBD.attributes().namedItem("id").nodeValue();
+					QString tmpKind = tmpMBD.attributes().namedItem("kind").nodeValue();
+					bool hasJsonApi = !tmpMBD.elementsByTagName("jsonapi").isEmpty();
+					if( tmpId == refid && tmpKind == "function" && hasJsonApi )
+					{
+						memberdef = tmpMBD;
+						break;
+					}
+				}
+
+				if(memberdef.isNull()) continue;
+
+				QString apiPath("/" + instanceName + "/" + methodName);
+				QString retvalType = memberdef.firstChildElement("type").text();
+				QMap<QString,MethodParam> paramsMap;
+				QStringList orderedParamNames;
+
+				QDomNodeList params = memberdef.elementsByTagName("param");
+				for (int k = 0; k < params.size(); ++k)
+				{
+					QDomElement tmpPE = params.item(k).toElement();
+					MethodParam tmpParam;
+					tmpParam.name = tmpPE.firstChildElement("declname").text();
+					QDomElement tmpType = tmpPE.firstChildElement("type");
+					QString& pType(tmpParam.type);
+					pType = tmpType.text();
+					if(pType.startsWith("const ")) pType.remove(0,6);
+					pType.replace(QString("&"), QString());
+					pType.replace(QString(" "), QString());
+					paramsMap.insert(tmpParam.name, tmpParam);
+					orderedParamNames.push_back(tmpParam.name);
+				}
+
+				QDomNodeList parameternames = memberdef.elementsByTagName("parametername");
+				for (int k = 0; k < parameternames.size(); ++k)
+				{
+					QDomElement tmpPN = parameternames.item(k).toElement();
+					MethodParam& tmpParam = paramsMap[tmpPN.text()];
+					QString tmpD = tmpPN.attributes().namedItem("direction").nodeValue();
+					tmpParam.in = tmpD.contains("in");
+					tmpParam.out = tmpD.contains("out");
+				}
+
+				qDebug() << instanceName << apiPath << retvalType << typeName
+				         << methodName;
+				for (const QString& pn : orderedParamNames)
+				{
+					const MethodParam& mp(paramsMap[pn]);
+					qDebug() << "\t" << mp.type << mp.name << mp.in << mp.out;
+				}
+
+				QString retvalSerialization;
+				if(retvalType != "void")
+					retvalSerialization = "\t\t\tRS_SERIAL_PROCESS(retval);";
+
+				QString paramsDeclaration;
+				QString inputParamsDeserialization;
+				QString outputParamsSerialization;
+				for (const QString& pn : orderedParamNames)
+				{
+					const MethodParam& mp(paramsMap[pn]);
+					paramsDeclaration += "\t\t" + mp.type + " " + mp.name + ";\n";
+					if(mp.in)
+						inputParamsDeserialization += "\t\t\tRS_SERIAL_PROCESS("
+						        + mp.name + ");\n";
+					if(mp.out)
+						outputParamsSerialization += "\t\t\tRS_SERIAL_PROCESS("
+						        + mp.name + ");\n";
+				}
+
+				QMap<QString,QString> substitutionsMap;
+				substitutionsMap.insert("instanceName", instanceName);
+				substitutionsMap.insert("methodName", methodName);
+				substitutionsMap.insert("paramsDeclaration", paramsDeclaration);
+				substitutionsMap.insert("inputParamsDeserialization", inputParamsDeserialization);
+				substitutionsMap.insert("outputParamsSerialization", outputParamsSerialization);
+				substitutionsMap.insert("retvalSerialization", retvalSerialization);
+				substitutionsMap.insert("retvalType", retvalType);
+				substitutionsMap.insert("callParamsList", orderedParamNames.join(", "));
+				substitutionsMap.insert("wrapperName", wrapperName);
+				substitutionsMap.insert("headerFileName", headerFileName);
+
+				QFile templFile(sourcePath + "/method-wrapper-template.cpp.tmpl");
+				templFile.open(QIODevice::ReadOnly);
+				QString wrapperDef(templFile.readAll());
+
+				QMap<QString,QString>::iterator it;
+				for ( it = substitutionsMap.begin();
+				      it != substitutionsMap.end(); ++it )
+					wrapperDef.replace(QString("$%"+it.key()+"%$"), QString(it.value()), Qt::CaseSensitive);
+
+				wrappersDefFile.write(wrapperDef.toLocal8Bit());
+
+				QString wrapperDecl("void " + instanceName + methodName + "Wrapper(const std::shared_ptr<rb::Session> session);\n");
+				wrappersDeclFile.write(wrapperDecl.toLocal8Bit());
+
+
+				QString wrapperReg("registerHandler(\""+apiPath+"\", "+wrapperName+");\n");
+				wrappersRegisterFile.write(wrapperReg.toLocal8Bit());
+			}
+		}
+	}
+
+	return 0;
+}

jsonapi-generator/src/jsonapi-generator.pro

@@ -0,0 +1,6 @@
+TARGET = jsonapi-generator
+
+QT *= core xml
+QT -= gui
+
+SOURCES += jsonapi-generator.cpp

jsonapi-generator/src/method-wrapper-template.cpp.tmpl

@@ -0,0 +1,79 @@
+/*
+ * RetroShare JSON API
+ * Copyright (C) 2018  Gioacchino Mazzurco <gio@eigenlab.org>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <sstream>
+#include <memory>
+#include <restbed>
+#include "retroshare/$%headerFileName%$"
+
+namespace rb = restbed;
+
+void $%wrapperName%$(const std::shared_ptr<rb::Session> session)
+{
+	size_t reqSize = session->get_request()->get_header("Content-Length", 0);
+	session->fetch( reqSize, [](
+	                const std::shared_ptr<rb::Session> session,
+	                const rb::Bytes& body )
+	{
+		RsGenericSerializer::SerializeContext cReq(
+		            nullptr, 0,
+		            RsGenericSerializer::SERIALIZATION_FLAG_YIELDING );
+		RsJson& jReq(cReq.mJson);
+		jReq.Parse(reinterpret_cast<const char*>(body.data()), body.size());
+
+		RsGenericSerializer::SerializeContext cAns;
+		RsJson& jAns(cAns.mJson);
+
+		// if caller specified caller_data put it back in the answhere
+		const char kcd[] = "caller_data";
+		if(jReq.HasMember(kcd))
+		    jAns.AddMember(kcd, jReq[kcd], jAns.GetAllocator());
+
+$%paramsDeclaration%$
+
+		// deserialize input parameters from JSON
+		{
+			RsGenericSerializer::SerializeContext& ctx(cReq);
+			RsGenericSerializer::SerializeJob j(RsGenericSerializer::FROM_JSON);
+$%inputParamsDeserialization%$
+		}
+
+		// call retroshare C++ API
+		$%retvalType%$ retval = $%instanceName%$->$%methodName%$($%callParamsList%$);
+
+		// serialize out parameters and return value to JSON
+		{
+			RsGenericSerializer::SerializeContext& ctx(cAns);
+			RsGenericSerializer::SerializeJob j(RsGenericSerializer::TO_JSON);
+$%retvalSerialization%$
+$%outputParamsSerialization%$
+		}
+
+		// return them to the API caller
+		std::stringstream ss;
+		ss << jAns;
+		std::string&& ans(ss.str());
+		const std::multimap<std::string, std::string> headers
+		{
+			{ "Content-Type", "text/json" },
+			{ "Content-Length", std::to_string(ans.length()) }
+		};
+		session->close(rb::OK, ans, headers);
+	} );
+}
+

libretroshare/src/jsonapi/jsonapi.cpp

@@ -0,0 +1,45 @@
+/*
+ * RetroShare JSON API
+ * Copyright (C) 2018  Gioacchino Mazzurco <gio@eigenlab.org>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "jsonapi.h"
+
+// Generated at compile time
+#include "jsonapi-wrappers.h"
+
+JsonApiServer::JsonApiServer(uint16_t port) : mPort(port)
+{
+// Generated at compile time
+#include "jsonapi-register.inl"
+}
+
+void JsonApiServer::run()
+{
+	std::shared_ptr<rb::Settings> settings(new rb::Settings);
+	settings->set_port(mPort);
+	settings->set_default_header("Connection", "close");
+	service.start(settings);
+}
+
+void JsonApiServer::registerHandler(const std::string& path, const std::function<void (const std::shared_ptr<restbed::Session>)>& handler)
+{
+	std::shared_ptr<restbed::Resource> resource(new rb::Resource);
+	resource->set_path(path);
+	resource->set_method_handler("GET", handler);
+	resource->set_method_handler("POST", handler);
+	service.publish(resource);
+}

libretroshare/src/jsonapi/jsonapi.h

@@ -0,0 +1,62 @@
+/*
+ * RetroShare JSON API
+ * Copyright (C) 2018  Gioacchino Mazzurco <gio@eigenlab.org>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <string>
+#include <memory>
+#include <restbed>
+#include <rapid_json/document.h>
+
+#include "retroshare/rsgxschannels.h"
+#include "serialiser/rstypeserializer.h"
+#include "util/rsthreads.h"
+
+namespace rb = restbed;
+
+void apiVersionHandler(const std::shared_ptr<rb::Session> session);
+void createChannelHandler(const std::shared_ptr<rb::Session> session);
+
+/**
+ * Simple usage
+ *  \code{.cpp}
+ *    JsonApiServer jas(9092);
+ *    jas.start("JsonApiServer");
+ *  \endcode
+ */
+struct JsonApiServer : RsSingleJobThread
+{
+	JsonApiServer(uint16_t port);
+
+	/// @see RsSingleJobThread
+	virtual void run();
+
+	/**
+	 * @param[in] handler function which will be called to handle the requested
+	 *   path, the function must be declared like:
+	 *   \code{.cpp}
+	 *     void functionName(const shared_ptr<restbed::Session> session)
+	 *   \endcode
+	 */
+	void registerHandler(
+	        const std::string& path,
+	        const std::function<void(const std::shared_ptr<rb::Session>)>& handler );
+
+private:
+	uint16_t mPort;
+	rb::Service service;
+};
+

libretroshare/src/libretroshare.pro

@@ -849,6 +849,64 @@ rs_gxs_trans {
     SOURCES += gxstrans/p3gxstransitems.cc gxstrans/p3gxstrans.cc
 }
 
+rs_jsonapi {
+    JSONAPI_GENERATOR_SRC=$$system_path($$clean_path($${RS_SRC_PATH}/jsonapi-generator/src/))
+    JSONAPI_GENERATOR_OUT=$$system_path($$clean_path($${RS_BUILD_PATH}/jsonapi-generator/src/))
+    JSONAPI_GENERATOR_EXE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-generator))
+    DOXIGEN_INPUT_DIRECTORY=$$system_path($$clean_path($${PWD}/retroshare/))
+    DOXIGEN_CONFIG_SRC=$$system_path($$clean_path($${RS_SRC_PATH}/jsonapi-generator/src/jsonapi-generator-doxygen.conf))
+    DOXIGEN_CONFIG_OUT=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-generator-doxygen.conf))
+    WRAPPERS_DEF_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-wrappers.cpp))
+    WRAPPERS_DECL_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-wrappers.h))
+    WRAPPERS_REG_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-register.inl))
+
+    restbed.target = $$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/librestbed.a))
+    restbed.commands = \
+        cd $${RS_SRC_PATH}; git submodule update --init --recursive;\
+        mkdir -p $${RESTBED_BUILD_PATH}; cd $${RESTBED_BUILD_PATH};\
+        cmake -DBUILD_SSL=OFF -DCMAKE_INSTALL_PREFIX=. -B. -H$${RESTBED_SRC_PATH};\
+        make; make install
+    QMAKE_EXTRA_TARGETS += restbed
+    libretroshare.depends += restbed
+    PRE_TARGETDEPS *= $${restbed.target}
+
+    PRE_TARGETDEPS *= $${JSONAPI_GENERATOR_EXE}
+    INCLUDEPATH *= $${JSONAPI_GENERATOR_OUT}
+    GENERATED_HEADERS += $${WRAPPERS_DECL_FILE} $${WRAPPERS_REG_FILE}
+    GENERATED_SOURCES += $${WRAPPERS_DEF_FILE}
+
+    jsonwrappersdecl.target = $${WRAPPERS_DECL_FILE}
+    jsonwrappersdecl.commands = \
+        cp $${DOXIGEN_CONFIG_SRC} $${DOXIGEN_CONFIG_OUT}; \
+        echo OUTPUT_DIRECTORY=$${JSONAPI_GENERATOR_OUT} >> $${DOXIGEN_CONFIG_OUT};\
+        echo INPUT=$${DOXIGEN_INPUT_DIRECTORY} >> $${DOXIGEN_CONFIG_OUT}; \
+        doxygen $${DOXIGEN_CONFIG_OUT}; \
+        $${JSONAPI_GENERATOR_EXE} $${JSONAPI_GENERATOR_SRC} $${JSONAPI_GENERATOR_OUT};
+    QMAKE_EXTRA_TARGETS += jsonwrappersdecl
+    libretroshare.depends += jsonwrappersdecl
+    PRE_TARGETDEPS *= $${WRAPPERS_DECL_FILE}
+
+    jsonwrappersdef.target = $${WRAPPERS_DEF_FILE}
+    jsonwrappersdef.commands = touch $${WRAPPERS_DEF_FILE}
+    jsonwrappersdef.depends = jsonwrappersdecl
+    QMAKE_EXTRA_TARGETS += jsonwrappersdef
+    libretroshare.depends += jsonwrappersdef
+    PRE_TARGETDEPS *= $${WRAPPERS_DEF_FILE}
+
+    jsonwrappersreg.target = $${WRAPPERS_REG_FILE}
+    jsonwrappersreg.commands = touch $${WRAPPERS_REG_FILE}
+    jsonwrappersreg.depends = jsonwrappersdef
+    QMAKE_EXTRA_TARGETS += jsonwrappersreg
+    libretroshare.depends += jsonwrappersreg
+    PRE_TARGETDEPS *= $${WRAPPERS_REG_FILE}
+
+    # Force recalculation of libretroshare dependencies see https://stackoverflow.com/a/47884045
+    QMAKE_EXTRA_TARGETS += libretroshare
+
+    HEADERS += jsonapi/jsonapi.h
+    SOURCES += jsonapi/jsonapi.cpp
+}
+
 
 
 

libretroshare/src/retroshare/rsgxschannels.h

@@ -6,7 +6,8 @@
  *
  * RetroShare C++ Interface.
  *
- * Copyright 2012-2012 by Robert Fernie.
+ * Copyright (C) 2012 by Robert Fernie.
+ * Copyright (C) 2018  Gioacchino Mazzurco <gio@eigenlab.org>
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Library General Public
@@ -33,35 +34,47 @@
 #include "retroshare/rstokenservice.h"
 #include "retroshare/rsgxsifacehelper.h"
 #include "retroshare/rsgxscommon.h"
+#include "serialiser/rsserializable.h"
 
 
-
-/* The Main Interface Class - for information about your Peers */
 class RsGxsChannels;
-extern RsGxsChannels *rsGxsChannels;
 
+/**
+ * Pointer to global instance of RsGxsChannels service implementation
+ * @jsonapi{development}
+ */
+extern RsGxsChannels* rsGxsChannels;
 
 // These should be in rsgxscommon.h
-
+struct RsGxsChannelGroup : RsSerializable
-class RsGxsChannelGroup
 {
-	public:
 	RsGroupMetaData mMeta;
 	std::string mDescription;
 	RsGxsImage mImage;
 
 	bool mAutoDownload;
+
+	/// @see RsSerializable
+	virtual void serial_process( RsGenericSerializer::SerializeJob j,
+	                             RsGenericSerializer::SerializeContext& ctx )
+	{
+		RS_SERIAL_PROCESS(mMeta);
+		RS_SERIAL_PROCESS(mDescription);
+		//RS_SERIAL_PROCESS(mImage);
+		RS_SERIAL_PROCESS(mAutoDownload);
+	}
 };
 
-class RsGxsChannelPost
+std::ostream &operator<<(std::ostream& out, const RsGxsChannelGroup& group);
+
+
+struct RsGxsChannelPost : RsSerializable
 {
-public:
 	RsGxsChannelPost() : mCount(0), mSize(0) {}
 
-public:
 	RsMsgMetaData mMeta;
 
-    std::set<RsGxsMessageId> mOlderVersions ;
+	std::set<RsGxsMessageId> mOlderVersions;
 	std::string mMsg;  // UTF8 encoded.
 
 	std::list<RsGxsFile> mFiles;
@@ -69,18 +82,31 @@ public:
 	uint64_t mSize;    // auto calced.
 
 	RsGxsImage mThumbnail;
+
+	/// @see RsSerializable
+	virtual void serial_process( RsGenericSerializer::SerializeJob j,
+	                             RsGenericSerializer::SerializeContext& ctx )
+	{
+		RS_SERIAL_PROCESS(mMeta);
+		RS_SERIAL_PROCESS(mOlderVersions);
+
+		RS_SERIAL_PROCESS(mMsg);
+		RS_SERIAL_PROCESS(mFiles);
+		RS_SERIAL_PROCESS(mCount);
+		RS_SERIAL_PROCESS(mSize);
+		//RS_SERIAL_PROCESS(mThumbnail);
+	}
 };
 
+std::ostream &operator<<(std::ostream& out, const RsGxsChannelPost& post);
 
-std::ostream &operator<<(std::ostream &out, const RsGxsChannelGroup &group);
-std::ostream &operator<<(std::ostream &out, const RsGxsChannelPost &post);
 
 class RsGxsChannels: public RsGxsIfaceHelper, public RsGxsCommentService
 {
-	public:
+public:
 
 	explicit RsGxsChannels(RsGxsIface *gxs)
-	  :RsGxsIfaceHelper(gxs)  {}
+	  :RsGxsIfaceHelper(gxs) {}
 	virtual ~RsGxsChannels() {}
 
 	/* Specific Service Data */
@@ -103,7 +129,16 @@ virtual bool setChannelAutoDownload(const RsGxsGroupId &groupId, bool enabled) =
 virtual bool getChannelAutoDownload(const RsGxsGroupId &groupid, bool& enabled) = 0;
 
 virtual bool setChannelDownloadDirectory(const RsGxsGroupId &groupId, const std::string& directory)=0;
-virtual bool getChannelDownloadDirectory(const RsGxsGroupId &groupId, std::string& directory)=0;
+
+	/**
+	 * Get download directory for the given channel
+	 * @jsonapi{development}
+	 * @param[in] channelId id of the channel
+	 * @param[out] directory reference to string where to store the path
+	 * @return false on error, true otherwise
+	 */
+	virtual bool getChannelDownloadDirectory( const RsGxsGroupId& channelId,
+	                                          std::string& directory ) = 0;
 
 //virtual void setChannelAutoDownload(uint32_t& token, const RsGxsGroupId& groupId, bool autoDownload) = 0;
 
@@ -113,19 +148,59 @@ virtual bool getChannelDownloadDirectory(const RsGxsGroupId &groupId, std::strin
 //virtual bool groupRestoreKeys(const std::string &groupId);
     virtual bool groupShareKeys(const RsGxsGroupId &groupId, std::set<RsPeerId>& peers)=0;
 
-	// Overloaded subscribe fn.
+	/**
-virtual bool subscribeToGroup(uint32_t &token, const RsGxsGroupId &groupId, bool subscribe) = 0;
+	 * @brief Request subscription to a group.
+	 * The action is performed asyncronously, so it could fail in a subsequent
+	 * phase even after returning true.
+	 * @jsonapi{development}
+	 * @param[out] token Storage for RsTokenService token to track request
+	 * status.
+	 * @param[in] groupId Channel id
+	 * @param[in] subscribe
+	 * @return false on error, true otherwise
+	 */
+	virtual bool subscribeToGroup( uint32_t& token, const RsGxsGroupId &groupId,
+	                               bool subscribe ) = 0;
 
-virtual bool createGroup(uint32_t &token, RsGxsChannelGroup &group) = 0;
+	/**
-virtual bool createPost(uint32_t &token, RsGxsChannelPost &post) = 0;
+	 * @brief Request channel creation.
+	 * The action is performed asyncronously, so it could fail in a subsequent
+	 * phase even after returning true.
+	 * @jsonapi{development}
+	 * @param[out] token Storage for RsTokenService token to track request
+	 * status.
+	 * @param[in] group Channel data (name, description...)
+	 * @return false on error, true otherwise
+	 */
+	virtual bool createGroup(uint32_t& token, RsGxsChannelGroup& group) = 0;
 
-virtual bool updateGroup(uint32_t &token, RsGxsChannelGroup &group) = 0;
+	/**
+	 * @brief Request post creation.
+	 * The action is performed asyncronously, so it could fail in a subsequent
+	 * phase even after returning true.
+	 * @jsonapi{development}
+	 * @param[out] token Storage for RsTokenService token to track request
+	 * status.
+	 * @param[in] post
+	 * @return false on error, true otherwise
+	 */
+	virtual bool createPost(uint32_t& token, RsGxsChannelPost& post) = 0;
+
+	/**
+	 * @brief Request channel change.
+	 * The action is performed asyncronously, so it could fail in a subsequent
+	 * phase even after returning true.
+	 * @jsonapi{development}
+	 * @param[out] token Storage for RsTokenService token to track request
+	 * status.
+	 * @param[in] group Channel data (name, description...) with modifications
+	 * @return false on error, true otherwise
+	 */
+	virtual bool updateGroup(uint32_t& token, RsGxsChannelGroup& group) = 0;
 
         // File Interface
 virtual bool ExtraFileHash(const std::string &path, std::string filename) = 0;
 virtual bool ExtraFileRemove(const RsFileHash &hash) = 0;
-
-
 };
 
 

libretroshare/src/retroshare/rsgxscommon.h

@@ -31,28 +31,39 @@
 #include <list>
 
 #include "rsgxsifacetypes.h"
+#include "serialiser/rsserializable.h"
 
-class RsGxsFile
+struct RsGxsFile : RsSerializable
 {
-	public:
 	RsGxsFile();
 	std::string mName;
 	RsFileHash mHash;
 	uint64_t    mSize;
-	//std::string mPath;
+
+	/// @see RsSerializable
+	virtual void serial_process( RsGenericSerializer::SerializeJob j,
+	                             RsGenericSerializer::SerializeContext& ctx )
+	{
+		RS_SERIAL_PROCESS(mName);
+		RS_SERIAL_PROCESS(mHash);
+		RS_SERIAL_PROCESS(mSize);
+	}
 };
 
-class RsGxsImage
+struct RsGxsImage
 {
-	public:
 	RsGxsImage();
 	~RsGxsImage();
-	RsGxsImage(const RsGxsImage& a); // TEMP use copy constructor and duplicate memory.
-RsGxsImage &operator=(const RsGxsImage &a); // Need this as well?
 
-//NB: Must make sure that we always use methods - to be consistent about malloc/free for this data.
+	/// Use copy constructor and duplicate memory.
-static uint8_t *allocate(uint32_t size);
+	RsGxsImage(const RsGxsImage& a);
-static void release(void *data);
+
+	RsGxsImage &operator=(const RsGxsImage &a); // Need this as well?
+
+	/** NB: Must make sure that we always use methods - to be consistent about
+	 * malloc/free for this data. */
+	static uint8_t *allocate(uint32_t size);
+	static void release(void *data);
 
 	void take(uint8_t *data, uint32_t size); // Copies Pointer.
 	void copy(uint8_t *data, uint32_t size); // Allocates and Copies.
@@ -61,7 +72,6 @@ static void release(void *data);
 
 	uint8_t *mData;
 	uint32_t mSize;
-
 };
 
 

libretroshare/src/retroshare/rsgxsifacetypes.h

@@ -124,7 +124,7 @@ struct RsGroupMetaData : RsSerializable
 
 
 
-struct RsMsgMetaData
+struct RsMsgMetaData : RsSerializable
 {
 	RsMsgMetaData() : mPublishTs(0), mMsgFlags(0), mMsgStatus(0), mChildTs(0) {}
 
@@ -155,6 +155,24 @@ struct RsMsgMetaData
     time_t      mChildTs;
     std::string mServiceString; // Service Specific Free-Form extra storage.
 
+	/// @see RsSerializable
+	virtual void serial_process( RsGenericSerializer::SerializeJob j,
+	                             RsGenericSerializer::SerializeContext& ctx )
+	{
+		RS_SERIAL_PROCESS(mGroupId);
+		RS_SERIAL_PROCESS(mMsgId);
+		RS_SERIAL_PROCESS(mThreadId);
+		RS_SERIAL_PROCESS(mParentId);
+		RS_SERIAL_PROCESS(mOrigMsgId);
+		RS_SERIAL_PROCESS(mAuthorId);
+		RS_SERIAL_PROCESS(mMsgName);
+		RS_SERIAL_PROCESS(mPublishTs);
+		RS_SERIAL_PROCESS(mMsgFlags);
+		RS_SERIAL_PROCESS(mMsgStatus);
+		RS_SERIAL_PROCESS(mChildTs);
+		RS_SERIAL_PROCESS(mServiceString);
+	}
+
     const std::ostream &print(std::ostream &out, std::string indent = "", std::string varName = "") const {
         out
             << indent << varName << " of RsMsgMetaData Values ###################" << std::endl

libretroshare/src/serialiser/rsserializer.cc

@@ -35,6 +35,7 @@ const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_NONE        ( 0
 const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_CONFIG      ( 0x0001 );
 const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_SIGNATURE   ( 0x0002 );
 const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_SKIP_HEADER ( 0x0004 );
+const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_YIELDING    ( 0x0008 );
 
 RsItem *RsServiceSerializer::deserialise(void *data, uint32_t *size)
 {

libretroshare/src/serialiser/rsserializer.h

@@ -225,7 +225,7 @@ struct RsGenericSerializer : RsSerialType
 		/** Allow shared allocator usage to avoid costly JSON deepcopy for
 		 *  nested RsSerializable */
 		SerializeContext(
-		        uint8_t *data, uint32_t size,
+		        uint8_t* data = nullptr, uint32_t size = 0,
 		        SerializationFlags flags = SERIALIZATION_FLAG_NONE,
 		        RsJson::AllocatorType* allocator = nullptr) :
 		    mData(data), mSize(size), mOffset(0), mOk(true), mFlags(flags),
@@ -260,6 +260,12 @@ struct RsGenericSerializer : RsSerialType
 	static const SerializationFlags SERIALIZATION_FLAG_SIGNATURE;      // 0x0002
 	static const SerializationFlags SERIALIZATION_FLAG_SKIP_HEADER;    // 0x0004
 
+	/** Used for JSON deserialization in JSON API, it causes the deserialization
+	 * to continue even if some field is missing (or incorrect), this way the
+	 * API is more user friendly as some methods need just part of the structs
+	 * they take as parameters. */
+	static const SerializationFlags SERIALIZATION_FLAG_YIELDING;       // 0x0008
+
 	/**
 	 * The following functions overload RsSerialType.
 	 * They *should not* need to be further overloaded.

libretroshare/src/serialiser/rstypeserializer.cc

@@ -42,7 +42,8 @@
 //static const uint32_t MAX_SERIALIZED_ARRAY_SIZE = 500 ;
 static const uint32_t MAX_SERIALIZED_CHUNK_SIZE = 10*1024*1024 ; // 10 MB.
 
-#define SAFE_GET_JSON_V() \
+#ifdef RSSERIAL_DEBUG
+#	define SAFE_GET_JSON_V() \
 	const char* mName = memberName.c_str(); \
 	bool ret = jDoc.HasMember(mName); \
 	if(!ret) \
@@ -50,10 +51,16 @@ static const uint32_t MAX_SERIALIZED_CHUNK_SIZE = 10*1024*1024 ; // 10 MB.
 	    std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName \
 	              << "\" not found in JSON:" << std::endl \
 	              << jDoc << std::endl << std::endl; \
-	    print_stacktrace(); \
 	    return false; \
 	} \
 	rapidjson::Value& v = jDoc[mName]
+#else // ifdef RSSERIAL_DEBUG
+#	define SAFE_GET_JSON_V() \
+	const char* mName = memberName.c_str(); \
+	bool ret = jDoc.HasMember(mName); \
+	if(!ret) return false; \
+	rapidjson::Value& v = jDoc[mName]
+#endif // ifdef RSSERIAL_DEBUG
 
 
 //============================================================================//

libretroshare/src/serialiser/rstypeserializer.h

@@ -57,9 +57,7 @@
     { \
 	    /* Use same allocator to avoid deep copy */\
 	    RsGenericSerializer::SerializeContext elCtx(\
-	                nullptr, 0, RsGenericSerializer::FORMAT_BINARY,\
+	                nullptr, 0, ctx.mFlags, &allocator );\
-	                RsGenericSerializer::SERIALIZATION_FLAG_NONE,\
-	                &allocator );\
 \
 	    /* If el is const the default serial_process template is matched */ \
 	    /* also when specialization is necessary so the compilation break */ \
@@ -85,7 +83,7 @@
 {\
 	using namespace rapidjson;\
 \
-	bool& ok(ctx.mOk);\
+	bool ok = ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;\
 	Document& jDoc(ctx.mJson);\
 	Document::AllocatorType& allocator = jDoc.GetAllocator();\
 \
@@ -100,19 +98,21 @@
 	    for (auto&& arrEl : jDoc[arrKey].GetArray())\
         {\
 	        RsGenericSerializer::SerializeContext elCtx(\
-	                    nullptr, 0, RsGenericSerializer::FORMAT_BINARY,\
+	                    nullptr, 0, ctx.mFlags, &allocator );\
-	                    RsGenericSerializer::SERIALIZATION_FLAG_NONE,\
-	                    &allocator );\
 	        elCtx.mJson.AddMember(arrKey, arrEl, allocator);\
 \
 	        T el;\
 	        serial_process(j, elCtx, el, memberName); \
 	        ok = ok && elCtx.mOk;\
 \
+	        ctx.mOk &= ok;\
 	        if(ok) v.INSERT_FUN(el);\
 	        else break;\
 	    }\
 	}\
+\
+	ctx.mOk = false;\
+\
 } while(false)
 
 std::ostream &operator<<(std::ostream &out, const RsJson &jDoc);
@@ -133,8 +133,8 @@ struct RsTypeSerializer
 	template<typename T>
 	typename std::enable_if<std::is_same<RsTlvItem,T>::value || !(std::is_base_of<RsSerializable,T>::value || std::is_enum<T>::value || std::is_base_of<RsTlvItem,T>::value)>::type
 	static /*void*/ serial_process( RsGenericSerializer::SerializeJob j,
-	                            RsGenericSerializer::SerializeContext& ctx,
+	                                RsGenericSerializer::SerializeContext& ctx,
-	                            T& member, const std::string& member_name)
+	                                T& member, const std::string& member_name )
 	{
 		switch(j)
 		{
@@ -156,7 +156,8 @@ struct RsTypeSerializer
 			ctx.mOk = ctx.mOk && to_JSON(member_name, member, ctx.mJson);
 			break;
 		case RsGenericSerializer::FROM_JSON:
-			ctx.mOk = ctx.mOk && from_JSON(member_name, member, ctx.mJson);
+			ctx.mOk &= (ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
+			        && from_JSON(member_name, member, ctx.mJson);
 			break;
 		default:
 			std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
@@ -194,8 +195,9 @@ struct RsTypeSerializer
 			        to_JSON(member_name, type_id, member, ctx.mJson);
 			break;
 		case RsGenericSerializer::FROM_JSON:
-			ctx.mOk = ctx.mOk &&
+			ctx.mOk &=
-			        from_JSON(member_name, type_id, member, ctx.mJson);
+			        (ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
+			        && from_JSON(member_name, type_id, member, ctx.mJson);
 			break;
 		default:
 			std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
@@ -287,15 +289,11 @@ struct RsTypeSerializer
 			{
 				// Use same allocator to avoid deep copy
 				RsGenericSerializer::SerializeContext kCtx(
-				            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
+				            nullptr, 0, ctx.mFlags, &allocator );
-				            RsGenericSerializer::SERIALIZATION_FLAG_NONE,
-				            &allocator );
 				serial_process<T>(j, kCtx, const_cast<T&>(kv.first), "key");
 
 				RsGenericSerializer::SerializeContext vCtx(
-				            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
+				            nullptr, 0, ctx.mFlags, &allocator );
-				            RsGenericSerializer::SERIALIZATION_FLAG_NONE,
-				            &allocator );
 				serial_process<U>(j, vCtx, const_cast<U&>(kv.second), "value");
 
 				if(kCtx.mOk && vCtx.mOk)
@@ -316,7 +314,7 @@ struct RsTypeSerializer
 		{
 			using namespace rapidjson;
 
-			bool& ok(ctx.mOk);
+			bool ok = ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;
 			Document& jDoc(ctx.mJson);
 			Document::AllocatorType& allocator = jDoc.GetAllocator();
 
@@ -336,9 +334,7 @@ struct RsTypeSerializer
 					if (!ok) break;
 
 					RsGenericSerializer::SerializeContext kCtx(
-					            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
+					            nullptr, 0, ctx.mFlags, &allocator );
-					            RsGenericSerializer::SERIALIZATION_FLAG_NONE,
-					            &allocator );
 					if(ok)
 						kCtx.mJson.AddMember("key", kvEl["key"], allocator);
 
@@ -346,9 +342,7 @@ struct RsTypeSerializer
 					ok = ok && (serial_process(j, kCtx, key, "key"), kCtx.mOk);
 
 					RsGenericSerializer::SerializeContext vCtx(
-					            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
+					            nullptr, 0, ctx.mFlags, &allocator );
-					            RsGenericSerializer::SERIALIZATION_FLAG_NONE,
-					            &allocator );
 					if(ok)
 						vCtx.mJson.AddMember("value", kvEl["value"], allocator);
 
@@ -356,14 +350,20 @@ struct RsTypeSerializer
 					ok = ok && ( serial_process(j, vCtx, value, "value"),
 					             vCtx.mOk );
 
+					ctx.mOk &= ok;
 					if(ok) v.insert(std::pair<T,U>(key,value));
 					else break;
 				}
 			}
 
+			ctx.mOk = false;
 			break;
 		}
-		default: break;
+		default:
+			std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
+			          << static_cast<std::underlying_type<decltype(j)>::type>(j)
+			          << std::endl;
+			exit(EINVAL);
 		}
 	}
 
@@ -566,7 +566,9 @@ struct RsTypeSerializer
 		case RsGenericSerializer::FROM_JSON:
 		{
 			uint32_t f;
-			ctx.mOk = from_JSON(memberName, f, ctx.mJson);
+			ctx.mOk &=
+			        (ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
+			        && from_JSON(memberName, f, ctx.mJson);
 			v = t_RsFlags32<N>(f);
 			break;
 		}
@@ -623,9 +625,7 @@ struct RsTypeSerializer
 
 			// Reuse allocator to avoid deep copy later
 			RsGenericSerializer::SerializeContext lCtx(
-			            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
+			            nullptr, 0, ctx.mFlags, &allocator );
-			            RsGenericSerializer::SERIALIZATION_FLAG_NONE,
-			            &allocator );
 
 			member.serial_process(j, lCtx);
 
@@ -642,28 +642,32 @@ struct RsTypeSerializer
 		}
 		case RsGenericSerializer::FROM_JSON:
 		{
-			rapidjson::Document& jDoc(ctx.mJson);
+			RsJson& jDoc(ctx.mJson);
 			const char* mName = memberName.c_str();
-			ctx.mOk = ctx.mOk && jDoc.HasMember(mName);
+			bool hasMember = jDoc.HasMember(mName);
+			bool yielding = ctx.mFlags &
+			        RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;
 
-			if(!ctx.mOk)
+			if(!hasMember)
 			{
-				std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName
+				if(!yielding)
-				          << "\" not found in JSON:" << std::endl
+				{
-				          << jDoc << std::endl << std::endl;
+					std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName
-				print_stacktrace();
+					          << "\" not found in JSON:" << std::endl
+					          << jDoc << std::endl << std::endl;
+					print_stacktrace();
+				}
+				ctx.mOk = false;
 				break;
 			}
 
 			rapidjson::Value& v = jDoc[mName];
 
-			RsGenericSerializer::SerializeContext lCtx(
+			RsGenericSerializer::SerializeContext lCtx(nullptr, 0, ctx.mFlags);
-			            nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
-			            RsGenericSerializer::SERIALIZATION_FLAG_NONE );
 			lCtx.mJson.SetObject() = v; // Beware of move semantic!!
 
 			member.serial_process(j, lCtx);
-			ctx.mOk = ctx.mOk && lCtx.mOk;
+			ctx.mOk &= lCtx.mOk;
 
 			break;
 		}

libretroshare/src/use_libretroshare.pri

@@ -17,6 +17,18 @@ sLibs =
 mLibs = $$RS_SQL_LIB ssl crypto $$RS_THREAD_LIB $$RS_UPNP_LIB
 dLibs =
 
+rs_jsonapi {
+    RS_SRC_PATH=$$system_path($$clean_path($${PWD}/../../))
+    RS_BUILD_PATH=$$system_path($$clean_path($${OUT_PWD}/../../))
+    RESTBED_SRC_PATH=$$system_path($$clean_path($${RS_SRC_PATH}/supportlibs/restbed))
+    RESTBED_BUILD_PATH=$$system_path($$clean_path($${RS_BUILD_PATH}/supportlibs/restbed))
+
+    INCLUDEPATH *= $$system_path($$clean_path($${RESTBED_BUILD_PATH}/include/))
+    QMAKE_LIBDIR *= $$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/))
+    # Using sLibs would fail as librestbed.a is generated at compile-time
+    LIBS *= -L$$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/)) -lrestbed
+}
+
 linux-* {
     mLibs += dl
 }

retroshare-gui/src/gui/settings/WebuiPage.cpp

@@ -22,6 +22,9 @@ resource_api::ApiServerMHD* WebuiPage::apiServerMHD = 0;
 resource_api::ApiServerLocal* WebuiPage::apiServerLocal = 0;
 #endif
 resource_api::RsControlModule* WebuiPage::controlModule = 0;
+#ifdef RS_JSONAPI
+JsonApiServer* WebuiPage::jsonApiServer = nullptr;
+#endif
 
 WebuiPage::WebuiPage(QWidget */*parent*/, Qt::WindowFlags /*flags*/)
 {
@@ -105,6 +108,10 @@ QString WebuiPage::helpText() const
 // TODO: LIBRESAPI_LOCAL_SERVER Move in appropriate place
 #ifdef LIBRESAPI_LOCAL_SERVER
 	apiServerLocal = new resource_api::ApiServerLocal(apiServer, resource_api::ApiServerLocal::serverPath());
+#endif
+#ifdef RS_JSONAPI
+	jsonApiServer = new JsonApiServer(9092);
+	jsonApiServer->start("WebuiPage::jsonApiServer");
 #endif
     return ok;
 }
@@ -126,6 +133,10 @@ QString WebuiPage::helpText() const
         delete controlModule;
         controlModule = 0;
     }
+#ifdef RS_JSONAPI
+	delete jsonApiServer;
+	jsonApiServer = nullptr;
+#endif
 }
 
 /*static*/ void WebuiPage::showWebui()

retroshare-gui/src/gui/settings/WebuiPage.h

@@ -3,6 +3,10 @@
 #include <retroshare-gui/configpage.h>
 #include "ui_WebuiPage.h"
 
+#ifdef RS_JSONAPI
+#	include "jsonapi/jsonapi.h"
+#endif
+
 namespace resource_api{
     class ApiServer;
     class ApiServerMHD;
@@ -55,4 +59,7 @@ private:
   static resource_api::ApiServerLocal* apiServerLocal;
  #endif
   static resource_api::RsControlModule* controlModule;
+#ifdef RS_JSONAPI
+  static JsonApiServer* jsonApiServer;
+#endif
 };

retroshare.pri

@@ -115,6 +115,12 @@ rs_macos10.9:CONFIG -= rs_macos10.11
 rs_macos10.10:CONFIG -= rs_macos10.11
 rs_macos10.12:CONFIG -= rs_macos10.11
 
+# To enable JSON API append the following assignation to qmake command line
+# "CONFIG+=rs_jsonapi"
+CONFIG *= no_rs_jsonapi
+rs_jsonapi:CONFIG -= no_rs_jsonapi
+
+
 ###########################################################################################################################################################
 #
 #  V07_NON_BACKWARD_COMPATIBLE_CHANGE_001:
@@ -157,7 +163,7 @@ rs_v07_changes {
 }
 
 ################################################################################
-## RetroShare qmake functions goes here as all the rest may use them ###########
+## RetroShare qmake functions goes here as all the rest may use them. ##########
 ################################################################################
 
 ## Qt versions older the 5 are not supported anymore, check if the user is
@@ -273,7 +279,12 @@ no_sqlcipher {
 
 rs_autologin {
     DEFINES *= RS_AUTOLOGIN
-    warning("You have enabled RetroShare auto-login, this is discouraged. The usage of auto-login on some linux distributions may allow someone having access to your session to steal the SSL keys of your node location and therefore compromise your security")
+    RS_AUTOLOGIN_WARNING_MSG = \
+        You have enabled RetroShare auto-login, this is discouraged. The usage \
+        of auto-login on some linux distributions may allow someone having \
+        access to your session to steal the SSL keys of your node location and \
+        therefore compromise your security
+    warning("$${RS_AUTOLOGIN_WARNING_MSG}")
 }
 
 rs_onlyhiddennode {
@@ -313,6 +324,10 @@ rs_chatserver {
     DEFINES *= RS_CHATSERVER
 }
 
+rs_jsonapi {
+    DEFINES *= RS_JSONAPI
+}
+
 debug {
     QMAKE_CXXFLAGS -= -O2 -fomit-frame-pointer
     QMAKE_CFLAGS -= -O2 -fomit-frame-pointer

supportlibs/restbed

@@ -0,0 +1 @@
+Subproject commit c27c6726d28c42e2e1b7537ba63eeb23e944789d