Git-Mirrors/RetroShare
1
0
Fork 0
mirror of https://github.com/RetroShare/RetroShare.git synced 2024-10-01 02:35:48 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity

Implement automatic JSON API generation

Browse Source 
qmake file add jsonapi-generator target to compile JSON API generator
qmake files add rs_jsonapi CONFIG option to enable/disable JSON API at compile
  time
RsTypeSerializer pass down same serialization flags when creating new context
  for nested objects serial job
RsGxsChannels expose a few methods through JSON API as example
Derive a few GXS types (RsGxsChannelGroup, RsGxsChannelPost, RsGxsFile,
  RsMsgMetaData) from RsSerializables so they can be used for the JSON API
Create RsGenericSerializer::SERIALIZATION_FLAG_YIELDING so JSON objects that
  miss some fields can be still deserialized, this improve API usability
SerializeContext offer friendly constructor with default paramethers
Add restbed 4.6 library as git submodule as most systems doesn't have it yet
Add a bit of documentation about JSON API into jsonapi-generator/README.adoc
Add JsonApiServer class to expose the JSON API via HTTP protocol
This commit is contained in:
Gioacchino Mazzurco 2018-06-23 17:13:38 +02:00
parent 2f159efb10
commit 7ad337c8d2
No known key found for this signature in database
GPG Key ID: A1FBCA3872E87051
22 changed files with 1205 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitmodules vendored Normal file
View File

@ -0,0 +1,3 @@
[submodule "supportlibs/restbed"]
path = supportlibs/restbed
url = https://github.com/Corvusoft/restbed.git

8
RetroShare.pro
View File

@ -10,7 +10,13 @@ retrotor {
} else {
SUBDIRS += libbitdht
libbitdht.file = libbitdht/src/libbitdht.pro
libretroshare.depends = openpgpsdk libbitdht
libretroshare.depends = openpgpsdk libbitdht
}
rs_jsonapi {
SUBDIRS += jsonapi-generator
jsonapi-generator.file = jsonapi-generator/src/jsonapi-generator.pro
libretroshare.depends += jsonapi-generator
}
SUBDIRS += libretroshare

227
jsonapi-generator/README.adoc Normal file
View File

@ -0,0 +1,227 @@
RetroShare JSON API
===================
:Cxx: C++
== How to use RetroShare JSON API
Look for methods marked with +@jsonapi+ doxygen custom command into
+libretroshare/src/retroshare+. The method path is composed by service instance
pointer name like +rsGxsChannels+ for +RsGxsChannels+, and the method name like
+createGroup+ and pass the input paramethers as a JSON object.
.paramethers.json
[source,json]
--------------------------------------------------------------------------------
{
"group":{
"mMeta":{
"mGroupName":"JSON test group",
"mGroupFlags":4,
"mSignFlags":520
},
"mDescription":"JSON test group description"
},
"caller_data":"Here can go any kind of JSON data (even objects) that the caller want to get back together with the response"
}
--------------------------------------------------------------------------------
.Calling the JSON API with curl on the terminal
[source,bash]
--------------------------------------------------------------------------------
curl -H "Content-Type: application/json" --data @paramethers.json \
http://127.0.0.1:9092/rsGxsChannels/createGroup
--------------------------------------------------------------------------------
.JSON API call result
[source,json]
--------------------------------------------------------------------------------
{
"caller_data": "Here can go any kind of JSON data (even objects) that the caller want to get back together with the response",
"retval": true,
"token": 3
}
--------------------------------------------------------------------------------
== Offer new RetroShare services through JSON API
To offer a retroshare service through the JSON API, first of all one need find
the global pointer to the service instance and document it in doxygen syntax,
plus marking with the custom doxygen command +@jsonapi{RS_VERSION}+ where
+RS_VERSION+ is the retroshare version in which this service became available
with the current semantic (major changes to the service semantic, changes the
meaning of the service itself, so the version should be updated in the
documentation in that case).
.Service instance pointer in rsgxschannels.h
[source,cpp]
--------------------------------------------------------------------------------
/**
* Pointer to global instance of RsGxsChannels service implementation
* @jsonapi{development}
*/
extern RsGxsChannels* rsGxsChannels;
--------------------------------------------------------------------------------
Once the service instance itself is known to the JSON API you need to document
in doxygen syntax and mark with the custom doxygen command
+@jsonapi{RS_VERSION}+ the methods of the service that you want to make
available through JSON API.
.Offering RsGxsChannels::getChannelDownloadDirectory in rsgxschannels.h
[source,cpp]
--------------------------------------------------------------------------------
/**
* Get download directory for the given channel
* @jsonapi{development}
* @param[in] channelId id of the channel
* @param[out] directory reference to string where to store the path
* @return false on error, true otherwise
*/
virtual bool getChannelDownloadDirectory( const RsGxsGroupId& channelId,
std::string& directory ) = 0;
--------------------------------------------------------------------------------
For each paramether you must specify if it is used as input +@param[in]+ as
output +@param[out]+ or both +@param[inout]+. Paramethers and return value
types must be of a type supported by +RsTypeSerializer+ which already support
most basic types (+bool+, +std::string+...), +RsSerializable+ and containers of
them like +std::vector<std::string>+. Paramethers passed by value and by
reference of those types are both supported, while passing by pointer is not
supported. If your paramether or return +class+/+struct+ type is not supported
yet by +RsTypeSerializer+ most convenient approach is to make it derive from
+RsSerializable+ and implement +serial_process+ method like I did with
+RsGxsChannelGroup+.
.Deriving RsGxsChannelGroup from RsSerializable in rsgxschannels.h
[source,cpp]
--------------------------------------------------------------------------------
struct RsGxsChannelGroup : RsSerializable
{
RsGroupMetaData mMeta;
std::string mDescription;
RsGxsImage mImage;
bool mAutoDownload;
/// @see RsSerializable
virtual void serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx )
{
RS_SERIAL_PROCESS(mMeta);
RS_SERIAL_PROCESS(mDescription);
//RS_SERIAL_PROCESS(mImage);
RS_SERIAL_PROCESS(mAutoDownload);
}
};
--------------------------------------------------------------------------------
You can do the same recursively for any member of your +struct+ that is not yet
supported by +RsTypeSerializer+ like I should have done for +RsGxsImage mImage;+
but didn't have done yet because the day is so beatiful and i want to spend some
time outside :D.
== A bit of history
=== First writings about this
The previous attempt of exposing a RetroShare JSON API is called +libresapi+ and
unfortunatley it requires a bunch of boilerplate code when we want to expose
something present in the {Cxx} API in the JSON API.
As an example here you can see the libresapi that exposes part of the retroshare
chat {Cxx} API and lot of boilerplate code just to convert {Cxx} objects to JSON
https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L44
To avoid the {Cxx} to JSON and back conversion boilerplate code I have worked out
an extension to our {Cxx} serialization code so it is capable to serialize and
deserialize to JSON you can see it in this pull request
https://github.com/RetroShare/RetroShare/pull/1155
So first step toward having a good API is to take advantage of the fact that RS
is now capable of converting C++ objects from and to JSON.
The current API is accessible via HTTP and unix socket, there is no
authentication in both of them, so anyone having access to the HTTP server or to
the unix socket can access the API without extra restrictions.
Expecially for the HTTP API this is a big risk because also if the http server
listen on 127.0.0.1 every application on the machine (even rogue javascript
running on your web browser) can access that and for example on android it is
not safe at all (because of that I implemented the unix socket access so at
least in android API was reasonably safe) because of this.
A second step to improve the API would be to implement some kind of API
authentication mechanism (it would be nice that the mechanism is handled at API
level and not at transport level so we can use it for any API trasport not just
HTTP for example)
The HTTP server used by libresapi is libmicrohttpd server that is very minimal,
it doesn't provide HTTPS nor modern HTTP goodies, like server notifications,
websockets etc. because the lack of support we have a token polling mechanism in
libresapi to avoid polling for every thing but it is still ugly, so if we can
completely get rid of polling in the API that would be really nice.
I have done a crawl to look for a replacement and briefly looked at
- https://www.gnu.org/software/libmicrohttpd/
- http://wolkykim.github.io/libasyncd/
- https://github.com/corvusoft/restbed
- https://code.facebook.com/posts/1503205539947302/introducing-proxygen-facebook-s-c-http-framework/
- https://github.com/cmouse/yahttp
taking in account a few metrics like modern HTTP goodies support, license,
platform support, external dependencies and documentation it seemed to me that
restbed is the more appropriate.
Another source of boilerplate code into libresapi is the mapping between JSON
API requests and C++ API methods as an example you can look at this
https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ChatHandler.cpp#L158
and this
https://github.com/RetroShare/RetroShare/blob/v0.6.4/libresapi/src/api/ApiServer.cpp#L253
The abstract logic of this thing is, when libreasapi get a request like
+/chat/initiate_distant_chat+ then call
+ChatHandler::handleInitiateDistantChatConnexion+ which in turn is just a
wrapper of +RsMsgs::initiateDistantChatConnexion+ all this process is basically
implemented as boilerplate code and would be unnecessary in a smarter design of
the API because almost all the information needed is already present in the
C++ API +libretroshare/src/retroshare+.
So a third step to improve the JSON API would be to remove this source of
boilerplate code by automatizing the mapping between C++ and JSON API call.
This may result a little tricky as language parsing or other adevanced things
may be required.
Hope this dive is useful for you +
Cheers +
G10h4ck
=== Second writings about this
I have been investigating a bit more about:
[verse, G10h4ck]
________________________________________________________________________________
So a third step to improve the JSON API would be to remove this source of
boilerplate code by automatizing the mapping between C++ and JSON API call
________________________________________________________________________________
After spending some hours investigating this topic the most reasonable approach
seems to:
1. Properly document headers in +libretroshare/src/retroshare/+ in doxygen syntax
specifying wihich params are input and/or output (doxygen sysntax for this is
+@param[in/out/inout]+) this will be the API documentation too.
2. At compile time use doxygen to generate XML description of the headers and use
the XML to generate the JSON api server stub.
http://www.stack.nl/~dimitri/doxygen/manual/customize.html#xmlgenerator
3. Enjoy

230
jsonapi-generator/src/jsonapi-generator-doxygen.conf Normal file
View File

@ -0,0 +1,230 @@
DOXYFILE_ENCODING = UTF-8
PROJECT_NAME = "libretroshare"
#OUTPUT_DIRECTORY =
ALIASES += jsonapi{1}="\xmlonly<jsonapi minversion=\"\1\"/>\endxmlonly"
# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the
# documentation from any documented member that it re-implements.
# The default value is: YES.
INHERIT_DOCS = YES
# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments
# according to the Markdown format, which allows for more readable
# documentation. See http://daringfireball.net/projects/markdown/ for details.
# The output of markdown processing is further processed by doxygen, so you can
# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in
# case of backward compatibilities issues.
# The default value is: YES.
MARKDOWN_SUPPORT = YES
# When enabled doxygen tries to link words that correspond to documented
# classes, or namespaces to their corresponding documentation. Such a link can
# be prevented in individual cases by putting a % sign in front of the word or
# globally by setting AUTOLINK_SUPPORT to NO.
# The default value is: YES.
AUTOLINK_SUPPORT = YES
# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in
# documentation are documented, even if no documentation was available. Private
# class members and static file members will be hidden unless the
# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES.
# Note: This will also disable the warnings about undocumented members that are
# normally produced when WARNINGS is set to YES.
# The default value is: NO.
EXTRACT_ALL = YES
# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all
# undocumented members inside documented classes or files. If set to NO these
# members will be included in the various overviews, but no documentation
# section is generated. This option has no effect if EXTRACT_ALL is enabled.
# The default value is: NO.
HIDE_UNDOC_MEMBERS = NO
# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all
# undocumented classes that are normally visible in the class hierarchy. If set
# to NO, these classes will be included in the various overviews. This option
# has no effect if EXTRACT_ALL is enabled.
# The default value is: NO.
HIDE_UNDOC_CLASSES = NO
#---------------------------------------------------------------------------
# Configuration options related to the input files
#---------------------------------------------------------------------------
# The INPUT tag is used to specify the files and/or directories that contain
# documented source files. You may enter file names like myfile.cpp or
# directories like /usr/src/myproject. Separate the files or directories with
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.
#INPUT =
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
# libiconv (or the iconv built into libc) for the transcoding. See the libiconv
# documentation (see: https://www.gnu.org/software/libiconv/) for the list of
# possible encodings.
# The default value is: UTF-8.
INPUT_ENCODING = UTF-8
# If the value of the INPUT tag contains directories, you can use the
# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and
# *.h) to filter out the source-files in the directories.
#
# Note that for custom extensions or not directly supported extensions you also
# need to set EXTENSION_MAPPING for the extension otherwise the files are not
# read by doxygen.
#
# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp,
# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h,
# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc,
# *.m, *.markdown, *.md, *.mm, *.dox, *.py, *.pyw, *.f90, *.f95, *.f03, *.f08,
# *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf and *.qsf.
FILE_PATTERNS = *.c \
*.cc \
*.cxx \
*.cpp \
*.c++ \
*.java \
*.ii \
*.ixx \
*.ipp \
*.i++ \
*.inl \
*.idl \
*.ddl \
*.odl \
*.h \
*.hh \
*.hxx \
*.hpp \
*.h++ \
*.cs \
*.d \
*.php \
*.php4 \
*.php5 \
*.phtml \
*.inc \
*.m \
*.markdown \
*.md \
*.mm \
*.dox \
*.py \
*.pyw \
*.f90 \
*.f95 \
*.f03 \
*.f08 \
*.f \
*.for \
*.tcl \
*.vhd \
*.vhdl \
*.ucf \
*.qsf
# The RECURSIVE tag can be used to specify whether or not subdirectories should
# be searched for input files as well.
# The default value is: NO.
RECURSIVE = YES
# The EXCLUDE tag can be used to specify files and/or directories that should be
# excluded from the INPUT source files. This way you can easily exclude a
# subdirectory from a directory tree whose root is specified with the INPUT tag.
#
# Note that relative paths are relative to the directory from which doxygen is
# run.
EXCLUDE =
# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or
# directories that are symbolic links (a Unix file system feature) are excluded
# from the input.
# The default value is: NO.
EXCLUDE_SYMLINKS = NO
# If the value of the INPUT tag contains directories, you can use the
# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude
# certain files from those directories.
#
# Note that the wildcards are matched against the file with absolute path, so to
# exclude all test directories for example use the pattern */test/*
EXCLUDE_PATTERNS =
# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names
# (namespaces, classes, functions, etc.) that should be excluded from the
# output. The symbol name can be a fully qualified name, a word, or if the
# wildcard * is used, a substring. Examples: ANamespace, AClass,
# AClass::ANamespace, ANamespace::*Test
#
# Note that the wildcards are matched against the file with absolute path, so to
# exclude all test directories use the pattern */test/*
EXCLUDE_SYMBOLS =
#---------------------------------------------------------------------------
# Configuration options related to the HTML output
#---------------------------------------------------------------------------
# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output
# The default value is: YES.
GENERATE_HTML = NO
# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
# The default value is: YES.
GENERATE_LATEX = NO
# If the GENERATE_XML tag is set to YES, doxygen will generate an XML file that
# captures the structure of the code including all documentation.
# The default value is: NO.
GENERATE_XML = YES
#---------------------------------------------------------------------------
# Configuration options related to the preprocessor
#---------------------------------------------------------------------------
# If the ENABLE_PREPROCESSING tag is set to YES, doxygen will evaluate all
# C-preprocessor directives found in the sources and include files.
# The default value is: YES.
ENABLE_PREPROCESSING = YES
# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names
# in the source code. If set to NO, only conditional compilation will be
# performed. Macro expansion can be done in a controlled way by setting
# EXPAND_ONLY_PREDEF to YES.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
MACRO_EXPANSION = NO
# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
# the macro expansion is limited to the macros specified with the PREDEFINED and
# EXPAND_AS_DEFINED tags.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_ONLY_PREDEF = NO

239
jsonapi-generator/src/jsonapi-generator.cpp Normal file
View File

@ -0,0 +1,239 @@
/*
* RetroShare JSON API
* Copyright (C) 2018 Gioacchino Mazzurco <gio@eigenlab.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <QCoreApplication>
#include <QDebug>
#include <QtXml>
#include <QDirIterator>
#include <QFileInfo>
#include <iterator>
struct MethodParam
{
MethodParam() : in(false), out(false) {}
QString type;
QString name;
bool in;
bool out;
};
int main(int argc, char *argv[])
{
if(argc != 3)
qFatal("Usage: jsonapi-generator SOURCE_PATH OUTPUT_PATH");
QString sourcePath(argv[1]);
QString outputPath(argv[2]);
QString doxPrefix(outputPath+"/xml/");
QString wrappersDefFilePath(outputPath + "/jsonapi-wrappers.cpp");
QFile wrappersDefFile(wrappersDefFilePath);
wrappersDefFile.remove();
if(!wrappersDefFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
qFatal(QString("Can't open: " + wrappersDefFilePath).toLatin1().data());
QString wrappersDeclFilePath(outputPath + "/jsonapi-wrappers.h");
QFile wrappersDeclFile(wrappersDeclFilePath);
wrappersDeclFile.remove();
if(!wrappersDeclFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
qFatal(QString("Can't open: " + wrappersDeclFilePath).toLatin1().data());
QString wrappersRegisterFilePath(outputPath + "/jsonapi-register.inl");
QFile wrappersRegisterFile(wrappersRegisterFilePath);
wrappersRegisterFile.remove();
if(!wrappersRegisterFile.open(QIODevice::WriteOnly|QIODevice::Append|QIODevice::Text))
qFatal(QString("Can't open: " + wrappersRegisterFilePath).toLatin1().data());
QDirIterator it(doxPrefix, QStringList() << "*8h.xml", QDir::Files);
while(it.hasNext())
{
QDomDocument hDoc;
QString hFilePath(it.next());
QString parseError; int line, column;
QFile hFile(hFilePath);
if (!hFile.open(QIODevice::ReadOnly) ||
!hDoc.setContent(&hFile, &parseError, &line, &column))
{
qWarning() << "Error parsing:" << hFilePath
<< parseError << line << column;
continue;
}
QFileInfo hfi(hFile);
QString headerFileName(hfi.fileName());
headerFileName.replace(QString("_8h.xml"), QString(".h"));
QDomNodeList sectiondefs = hDoc.elementsByTagName("sectiondef");
for(int j = 0; j < sectiondefs.size(); ++j)
{
QDomElement sectDef = sectiondefs.item(j).toElement();
if( sectDef.attributes().namedItem("kind").nodeValue() != "var"
|| sectDef.elementsByTagName("jsonapi").isEmpty() )
continue;
QString instanceName =
sectDef.elementsByTagName("name").item(0).toElement().text();
QString typeName =
sectDef.elementsByTagName("ref").item(0).toElement().text();
QString typeFilePath(doxPrefix);
typeFilePath += sectDef.elementsByTagName("ref").item(0)
.attributes().namedItem("refid").nodeValue();
typeFilePath += ".xml";
QDomDocument typeDoc;
QFile typeFile(typeFilePath);
if ( !typeFile.open(QIODevice::ReadOnly) ||
!typeDoc.setContent(&typeFile, &parseError, &line, &column) )
{
qWarning() << "Error parsing:" << typeFilePath
<< parseError << line << column;
continue;
}
QDomNodeList members = typeDoc.elementsByTagName("member");
for (int i = 0; i < members.size(); ++i)
{
QDomNode member = members.item(i);
QString refid(member.attributes().namedItem("refid").nodeValue());
QString methodName(member.firstChildElement("name").toElement().text());
QString wrapperName(instanceName+methodName+"Wrapper");
QDomDocument defDoc;
QString defFilePath(doxPrefix + refid.split('_')[0] + ".xml");
QFile defFile(defFilePath);
if ( !defFile.open(QIODevice::ReadOnly) ||
!defDoc.setContent(&defFile, &parseError, &line, &column) )
{
qWarning() << "Error parsing:" << defFilePath
<< parseError << line << column;
continue;
}
QDomElement memberdef;
QDomNodeList memberdefs = typeDoc.elementsByTagName("memberdef");
for (int k = 0; k < memberdefs.size(); ++k)
{
QDomElement tmpMBD = memberdefs.item(k).toElement();
QString tmpId = tmpMBD.attributes().namedItem("id").nodeValue();
QString tmpKind = tmpMBD.attributes().namedItem("kind").nodeValue();
bool hasJsonApi = !tmpMBD.elementsByTagName("jsonapi").isEmpty();
if( tmpId == refid && tmpKind == "function" && hasJsonApi )
{
memberdef = tmpMBD;
break;
}
}
if(memberdef.isNull()) continue;
QString apiPath("/" + instanceName + "/" + methodName);
QString retvalType = memberdef.firstChildElement("type").text();
QMap<QString,MethodParam> paramsMap;
QStringList orderedParamNames;
QDomNodeList params = memberdef.elementsByTagName("param");
for (int k = 0; k < params.size(); ++k)
{
QDomElement tmpPE = params.item(k).toElement();
MethodParam tmpParam;
tmpParam.name = tmpPE.firstChildElement("declname").text();
QDomElement tmpType = tmpPE.firstChildElement("type");
QString& pType(tmpParam.type);
pType = tmpType.text();
if(pType.startsWith("const ")) pType.remove(0,6);
pType.replace(QString("&"), QString());
pType.replace(QString(" "), QString());
paramsMap.insert(tmpParam.name, tmpParam);
orderedParamNames.push_back(tmpParam.name);
}
QDomNodeList parameternames = memberdef.elementsByTagName("parametername");
for (int k = 0; k < parameternames.size(); ++k)
{
QDomElement tmpPN = parameternames.item(k).toElement();
MethodParam& tmpParam = paramsMap[tmpPN.text()];
QString tmpD = tmpPN.attributes().namedItem("direction").nodeValue();
tmpParam.in = tmpD.contains("in");
tmpParam.out = tmpD.contains("out");
}
qDebug() << instanceName << apiPath << retvalType << typeName
<< methodName;
for (const QString& pn : orderedParamNames)
{
const MethodParam& mp(paramsMap[pn]);
qDebug() << "\t" << mp.type << mp.name << mp.in << mp.out;
}
QString retvalSerialization;
if(retvalType != "void")
retvalSerialization = "\t\t\tRS_SERIAL_PROCESS(retval);";
QString paramsDeclaration;
QString inputParamsDeserialization;
QString outputParamsSerialization;
for (const QString& pn : orderedParamNames)
{
const MethodParam& mp(paramsMap[pn]);
paramsDeclaration += "\t\t" + mp.type + " " + mp.name + ";\n";
if(mp.in)
inputParamsDeserialization += "\t\t\tRS_SERIAL_PROCESS("
+ mp.name + ");\n";
if(mp.out)
outputParamsSerialization += "\t\t\tRS_SERIAL_PROCESS("
+ mp.name + ");\n";
}
QMap<QString,QString> substitutionsMap;
substitutionsMap.insert("instanceName", instanceName);
substitutionsMap.insert("methodName", methodName);
substitutionsMap.insert("paramsDeclaration", paramsDeclaration);
substitutionsMap.insert("inputParamsDeserialization", inputParamsDeserialization);
substitutionsMap.insert("outputParamsSerialization", outputParamsSerialization);
substitutionsMap.insert("retvalSerialization", retvalSerialization);
substitutionsMap.insert("retvalType", retvalType);
substitutionsMap.insert("callParamsList", orderedParamNames.join(", "));
substitutionsMap.insert("wrapperName", wrapperName);
substitutionsMap.insert("headerFileName", headerFileName);
QFile templFile(sourcePath + "/method-wrapper-template.cpp.tmpl");
templFile.open(QIODevice::ReadOnly);
QString wrapperDef(templFile.readAll());
QMap<QString,QString>::iterator it;
for ( it = substitutionsMap.begin();
it != substitutionsMap.end(); ++it )
wrapperDef.replace(QString("$%"+it.key()+"%$"), QString(it.value()), Qt::CaseSensitive);
wrappersDefFile.write(wrapperDef.toLocal8Bit());
QString wrapperDecl("void " + instanceName + methodName + "Wrapper(const std::shared_ptr<rb::Session> session);\n");
wrappersDeclFile.write(wrapperDecl.toLocal8Bit());
QString wrapperReg("registerHandler(\""+apiPath+"\", "+wrapperName+");\n");
wrappersRegisterFile.write(wrapperReg.toLocal8Bit());
}
}
}
return 0;
}

6
jsonapi-generator/src/jsonapi-generator.pro Normal file
View File

@ -0,0 +1,6 @@
TARGET = jsonapi-generator
QT *= core xml
QT -= gui
SOURCES += jsonapi-generator.cpp

79
jsonapi-generator/src/method-wrapper-template.cpp.tmpl Normal file
View File

@ -0,0 +1,79 @@
/*
* RetroShare JSON API
* Copyright (C) 2018 Gioacchino Mazzurco <gio@eigenlab.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <sstream>
#include <memory>
#include <restbed>
#include "retroshare/$%headerFileName%$"
namespace rb = restbed;
void $%wrapperName%$(const std::shared_ptr<rb::Session> session)
{
size_t reqSize = session->get_request()->get_header("Content-Length", 0);
session->fetch( reqSize, [](
const std::shared_ptr<rb::Session> session,
const rb::Bytes& body )
{
RsGenericSerializer::SerializeContext cReq(
nullptr, 0,
RsGenericSerializer::SERIALIZATION_FLAG_YIELDING );
RsJson& jReq(cReq.mJson);
jReq.Parse(reinterpret_cast<const char*>(body.data()), body.size());
RsGenericSerializer::SerializeContext cAns;
RsJson& jAns(cAns.mJson);
// if caller specified caller_data put it back in the answhere
const char kcd[] = "caller_data";
if(jReq.HasMember(kcd))
jAns.AddMember(kcd, jReq[kcd], jAns.GetAllocator());
$%paramsDeclaration%$
// deserialize input parameters from JSON
{
RsGenericSerializer::SerializeContext& ctx(cReq);
RsGenericSerializer::SerializeJob j(RsGenericSerializer::FROM_JSON);
$%inputParamsDeserialization%$
}
// call retroshare C++ API
$%retvalType%$ retval = $%instanceName%$->$%methodName%$($%callParamsList%$);
// serialize out parameters and return value to JSON
{
RsGenericSerializer::SerializeContext& ctx(cAns);
RsGenericSerializer::SerializeJob j(RsGenericSerializer::TO_JSON);
$%retvalSerialization%$
$%outputParamsSerialization%$
}
// return them to the API caller
std::stringstream ss;
ss << jAns;
std::string&& ans(ss.str());
const std::multimap<std::string, std::string> headers
{
{ "Content-Type", "text/json" },
{ "Content-Length", std::to_string(ans.length()) }
};
session->close(rb::OK, ans, headers);
} );
}

45
libretroshare/src/jsonapi/jsonapi.cpp Normal file
View File

@ -0,0 +1,45 @@
/*
* RetroShare JSON API
* Copyright (C) 2018 Gioacchino Mazzurco <gio@eigenlab.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "jsonapi.h"
// Generated at compile time
#include "jsonapi-wrappers.h"
JsonApiServer::JsonApiServer(uint16_t port) : mPort(port)
{
// Generated at compile time
#include "jsonapi-register.inl"
}
void JsonApiServer::run()
{
std::shared_ptr<rb::Settings> settings(new rb::Settings);
settings->set_port(mPort);
settings->set_default_header("Connection", "close");
service.start(settings);
}
void JsonApiServer::registerHandler(const std::string& path, const std::function<void (const std::shared_ptr<restbed::Session>)>& handler)
{
std::shared_ptr<restbed::Resource> resource(new rb::Resource);
resource->set_path(path);
resource->set_method_handler("GET", handler);
resource->set_method_handler("POST", handler);
service.publish(resource);
}

62
libretroshare/src/jsonapi/jsonapi.h Normal file
View File

@ -0,0 +1,62 @@
/*
* RetroShare JSON API
* Copyright (C) 2018 Gioacchino Mazzurco <gio@eigenlab.org>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <string>
#include <memory>
#include <restbed>
#include <rapid_json/document.h>
#include "retroshare/rsgxschannels.h"
#include "serialiser/rstypeserializer.h"
#include "util/rsthreads.h"
namespace rb = restbed;
void apiVersionHandler(const std::shared_ptr<rb::Session> session);
void createChannelHandler(const std::shared_ptr<rb::Session> session);
/**
* Simple usage
* \code{.cpp}
* JsonApiServer jas(9092);
* jas.start("JsonApiServer");
* \endcode
*/
struct JsonApiServer : RsSingleJobThread
{
JsonApiServer(uint16_t port);
/// @see RsSingleJobThread
virtual void run();
/**
* @param[in] handler function which will be called to handle the requested
* path, the function must be declared like:
* \code{.cpp}
* void functionName(const shared_ptr<restbed::Session> session)
* \endcode
*/
void registerHandler(
const std::string& path,
const std::function<void(const std::shared_ptr<rb::Session>)>& handler );
private:
uint16_t mPort;
rb::Service service;
};

58
libretroshare/src/libretroshare.pro
View File

@ -849,6 +849,64 @@ rs_gxs_trans {
SOURCES += gxstrans/p3gxstransitems.cc gxstrans/p3gxstrans.cc
}
rs_jsonapi {
JSONAPI_GENERATOR_SRC=$$system_path($$clean_path($${RS_SRC_PATH}/jsonapi-generator/src/))
JSONAPI_GENERATOR_OUT=$$system_path($$clean_path($${RS_BUILD_PATH}/jsonapi-generator/src/))
JSONAPI_GENERATOR_EXE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-generator))
DOXIGEN_INPUT_DIRECTORY=$$system_path($$clean_path($${PWD}/retroshare/))
DOXIGEN_CONFIG_SRC=$$system_path($$clean_path($${RS_SRC_PATH}/jsonapi-generator/src/jsonapi-generator-doxygen.conf))
DOXIGEN_CONFIG_OUT=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-generator-doxygen.conf))
WRAPPERS_DEF_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-wrappers.cpp))
WRAPPERS_DECL_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-wrappers.h))
WRAPPERS_REG_FILE=$$system_path($$clean_path($${JSONAPI_GENERATOR_OUT}/jsonapi-register.inl))
restbed.target = $$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/librestbed.a))
restbed.commands = \
cd $${RS_SRC_PATH}; git submodule update --init --recursive;\
mkdir -p $${RESTBED_BUILD_PATH}; cd $${RESTBED_BUILD_PATH};\
cmake -DBUILD_SSL=OFF -DCMAKE_INSTALL_PREFIX=. -B. -H$${RESTBED_SRC_PATH};\
make; make install
QMAKE_EXTRA_TARGETS += restbed
libretroshare.depends += restbed
PRE_TARGETDEPS *= $${restbed.target}
PRE_TARGETDEPS *= $${JSONAPI_GENERATOR_EXE}
INCLUDEPATH *= $${JSONAPI_GENERATOR_OUT}
GENERATED_HEADERS += $${WRAPPERS_DECL_FILE} $${WRAPPERS_REG_FILE}
GENERATED_SOURCES += $${WRAPPERS_DEF_FILE}
jsonwrappersdecl.target = $${WRAPPERS_DECL_FILE}
jsonwrappersdecl.commands = \
cp $${DOXIGEN_CONFIG_SRC} $${DOXIGEN_CONFIG_OUT}; \
echo OUTPUT_DIRECTORY=$${JSONAPI_GENERATOR_OUT} >> $${DOXIGEN_CONFIG_OUT};\
echo INPUT=$${DOXIGEN_INPUT_DIRECTORY} >> $${DOXIGEN_CONFIG_OUT}; \
doxygen $${DOXIGEN_CONFIG_OUT}; \
$${JSONAPI_GENERATOR_EXE} $${JSONAPI_GENERATOR_SRC} $${JSONAPI_GENERATOR_OUT};
QMAKE_EXTRA_TARGETS += jsonwrappersdecl
libretroshare.depends += jsonwrappersdecl
PRE_TARGETDEPS *= $${WRAPPERS_DECL_FILE}
jsonwrappersdef.target = $${WRAPPERS_DEF_FILE}
jsonwrappersdef.commands = touch $${WRAPPERS_DEF_FILE}
jsonwrappersdef.depends = jsonwrappersdecl
QMAKE_EXTRA_TARGETS += jsonwrappersdef
libretroshare.depends += jsonwrappersdef
PRE_TARGETDEPS *= $${WRAPPERS_DEF_FILE}
jsonwrappersreg.target = $${WRAPPERS_REG_FILE}
jsonwrappersreg.commands = touch $${WRAPPERS_REG_FILE}
jsonwrappersreg.depends = jsonwrappersdef
QMAKE_EXTRA_TARGETS += jsonwrappersreg
libretroshare.depends += jsonwrappersreg
PRE_TARGETDEPS *= $${WRAPPERS_REG_FILE}
# Force recalculation of libretroshare dependencies see https://stackoverflow.com/a/47884045
QMAKE_EXTRA_TARGETS += libretroshare
HEADERS += jsonapi/jsonapi.h
SOURCES += jsonapi/jsonapi.cpp
}

121
libretroshare/src/retroshare/rsgxschannels.h
View File

@ -6,7 +6,8 @@
*
* RetroShare C++ Interface.
*
* Copyright 2012-2012 by Robert Fernie.
* Copyright (C) 2012 by Robert Fernie.
* Copyright (C) 2018 Gioacchino Mazzurco <gio@eigenlab.org>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
@ -33,35 +34,47 @@
#include "retroshare/rstokenservice.h"
#include "retroshare/rsgxsifacehelper.h"
#include "retroshare/rsgxscommon.h"
#include "serialiser/rsserializable.h"
/* The Main Interface Class - for information about your Peers */
class RsGxsChannels;
extern RsGxsChannels *rsGxsChannels;
/**
* Pointer to global instance of RsGxsChannels service implementation
* @jsonapi{development}
*/
extern RsGxsChannels* rsGxsChannels;
// These should be in rsgxscommon.h
class RsGxsChannelGroup
struct RsGxsChannelGroup : RsSerializable
{
public:
RsGroupMetaData mMeta;
std::string mDescription;
RsGxsImage mImage;
bool mAutoDownload;
/// @see RsSerializable
virtual void serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx )
{
RS_SERIAL_PROCESS(mMeta);
RS_SERIAL_PROCESS(mDescription);
//RS_SERIAL_PROCESS(mImage);
RS_SERIAL_PROCESS(mAutoDownload);
}
};
class RsGxsChannelPost
std::ostream &operator<<(std::ostream& out, const RsGxsChannelGroup& group);
struct RsGxsChannelPost : RsSerializable
{
public:
RsGxsChannelPost() : mCount(0), mSize(0) {}
public:
RsMsgMetaData mMeta;
std::set<RsGxsMessageId> mOlderVersions ;
std::set<RsGxsMessageId> mOlderVersions;
std::string mMsg; // UTF8 encoded.
std::list<RsGxsFile> mFiles;
@ -69,18 +82,31 @@ public:
uint64_t mSize; // auto calced.
RsGxsImage mThumbnail;
/// @see RsSerializable
virtual void serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx )
{
RS_SERIAL_PROCESS(mMeta);
RS_SERIAL_PROCESS(mOlderVersions);
RS_SERIAL_PROCESS(mMsg);
RS_SERIAL_PROCESS(mFiles);
RS_SERIAL_PROCESS(mCount);
RS_SERIAL_PROCESS(mSize);
//RS_SERIAL_PROCESS(mThumbnail);
}
};
std::ostream &operator<<(std::ostream& out, const RsGxsChannelPost& post);
std::ostream &operator<<(std::ostream &out, const RsGxsChannelGroup &group);
std::ostream &operator<<(std::ostream &out, const RsGxsChannelPost &post);
class RsGxsChannels: public RsGxsIfaceHelper, public RsGxsCommentService
{
public:
public:
explicit RsGxsChannels(RsGxsIface *gxs)
:RsGxsIfaceHelper(gxs) {}
:RsGxsIfaceHelper(gxs) {}
virtual ~RsGxsChannels() {}
/* Specific Service Data */
@ -103,7 +129,16 @@ virtual bool setChannelAutoDownload(const RsGxsGroupId &groupId, bool enabled) =
virtual bool getChannelAutoDownload(const RsGxsGroupId &groupid, bool& enabled) = 0;
virtual bool setChannelDownloadDirectory(const RsGxsGroupId &groupId, const std::string& directory)=0;
virtual bool getChannelDownloadDirectory(const RsGxsGroupId &groupId, std::string& directory)=0;
/**
* Get download directory for the given channel
* @jsonapi{development}
* @param[in] channelId id of the channel
* @param[out] directory reference to string where to store the path
* @return false on error, true otherwise
*/
virtual bool getChannelDownloadDirectory( const RsGxsGroupId& channelId,
std::string& directory ) = 0;
//virtual void setChannelAutoDownload(uint32_t& token, const RsGxsGroupId& groupId, bool autoDownload) = 0;
@ -113,19 +148,59 @@ virtual bool getChannelDownloadDirectory(const RsGxsGroupId &groupId, std::strin
//virtual bool groupRestoreKeys(const std::string &groupId);
virtual bool groupShareKeys(const RsGxsGroupId &groupId, std::set<RsPeerId>& peers)=0;
// Overloaded subscribe fn.
virtual bool subscribeToGroup(uint32_t &token, const RsGxsGroupId &groupId, bool subscribe) = 0;
/**
* @brief Request subscription to a group.
* The action is performed asyncronously, so it could fail in a subsequent
* phase even after returning true.
* @jsonapi{development}
* @param[out] token Storage for RsTokenService token to track request
* status.
* @param[in] groupId Channel id
* @param[in] subscribe
* @return false on error, true otherwise
*/
virtual bool subscribeToGroup( uint32_t& token, const RsGxsGroupId &groupId,
bool subscribe ) = 0;
virtual bool createGroup(uint32_t &token, RsGxsChannelGroup &group) = 0;
virtual bool createPost(uint32_t &token, RsGxsChannelPost &post) = 0;
/**
* @brief Request channel creation.
* The action is performed asyncronously, so it could fail in a subsequent
* phase even after returning true.
* @jsonapi{development}
* @param[out] token Storage for RsTokenService token to track request
* status.
* @param[in] group Channel data (name, description...)
* @return false on error, true otherwise
*/
virtual bool createGroup(uint32_t& token, RsGxsChannelGroup& group) = 0;
virtual bool updateGroup(uint32_t &token, RsGxsChannelGroup &group) = 0;
/**
* @brief Request post creation.
* The action is performed asyncronously, so it could fail in a subsequent
* phase even after returning true.
* @jsonapi{development}
* @param[out] token Storage for RsTokenService token to track request
* status.
* @param[in] post
* @return false on error, true otherwise
*/
virtual bool createPost(uint32_t& token, RsGxsChannelPost& post) = 0;
/**
* @brief Request channel change.
* The action is performed asyncronously, so it could fail in a subsequent
* phase even after returning true.
* @jsonapi{development}
* @param[out] token Storage for RsTokenService token to track request
* status.
* @param[in] group Channel data (name, description...) with modifications
* @return false on error, true otherwise
*/
virtual bool updateGroup(uint32_t& token, RsGxsChannelGroup& group) = 0;
// File Interface
virtual bool ExtraFileHash(const std::string &path, std::string filename) = 0;
virtual bool ExtraFileRemove(const RsFileHash &hash) = 0;
};

34
libretroshare/src/retroshare/rsgxscommon.h
View File

@ -31,28 +31,39 @@
#include <list>
#include "rsgxsifacetypes.h"
#include "serialiser/rsserializable.h"
class RsGxsFile
struct RsGxsFile : RsSerializable
{
public:
RsGxsFile();
std::string mName;
RsFileHash mHash;
uint64_t mSize;
//std::string mPath;
/// @see RsSerializable
virtual void serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx )
{
RS_SERIAL_PROCESS(mName);
RS_SERIAL_PROCESS(mHash);
RS_SERIAL_PROCESS(mSize);
}
};
class RsGxsImage
struct RsGxsImage
{
public:
RsGxsImage();
RsGxsImage();
~RsGxsImage();
RsGxsImage(const RsGxsImage& a); // TEMP use copy constructor and duplicate memory.
RsGxsImage &operator=(const RsGxsImage &a); // Need this as well?
//NB: Must make sure that we always use methods - to be consistent about malloc/free for this data.
static uint8_t *allocate(uint32_t size);
static void release(void *data);
/// Use copy constructor and duplicate memory.
RsGxsImage(const RsGxsImage& a);
RsGxsImage &operator=(const RsGxsImage &a); // Need this as well?
/** NB: Must make sure that we always use methods - to be consistent about
* malloc/free for this data. */
static uint8_t *allocate(uint32_t size);
static void release(void *data);
void take(uint8_t *data, uint32_t size); // Copies Pointer.
void copy(uint8_t *data, uint32_t size); // Allocates and Copies.
@ -61,7 +72,6 @@ static void release(void *data);
uint8_t *mData;
uint32_t mSize;
};

20
libretroshare/src/retroshare/rsgxsifacetypes.h
View File

@ -124,7 +124,7 @@ struct RsGroupMetaData : RsSerializable
struct RsMsgMetaData
struct RsMsgMetaData : RsSerializable
{
RsMsgMetaData() : mPublishTs(0), mMsgFlags(0), mMsgStatus(0), mChildTs(0) {}
@ -155,6 +155,24 @@ struct RsMsgMetaData
time_t mChildTs;
std::string mServiceString; // Service Specific Free-Form extra storage.
/// @see RsSerializable
virtual void serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx )
{
RS_SERIAL_PROCESS(mGroupId);
RS_SERIAL_PROCESS(mMsgId);
RS_SERIAL_PROCESS(mThreadId);
RS_SERIAL_PROCESS(mParentId);
RS_SERIAL_PROCESS(mOrigMsgId);
RS_SERIAL_PROCESS(mAuthorId);
RS_SERIAL_PROCESS(mMsgName);
RS_SERIAL_PROCESS(mPublishTs);
RS_SERIAL_PROCESS(mMsgFlags);
RS_SERIAL_PROCESS(mMsgStatus);
RS_SERIAL_PROCESS(mChildTs);
RS_SERIAL_PROCESS(mServiceString);
}
const std::ostream &print(std::ostream &out, std::string indent = "", std::string varName = "") const {
out
<< indent << varName << " of RsMsgMetaData Values ###################" << std::endl

1
libretroshare/src/serialiser/rsserializer.cc
View File

@ -35,6 +35,7 @@ const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_NONE ( 0
const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_CONFIG ( 0x0001 );
const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_SIGNATURE ( 0x0002 );
const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_SKIP_HEADER ( 0x0004 );
const SerializationFlags RsGenericSerializer::SERIALIZATION_FLAG_YIELDING ( 0x0008 );
RsItem *RsServiceSerializer::deserialise(void *data, uint32_t *size)
{

8
libretroshare/src/serialiser/rsserializer.h
View File

@ -225,7 +225,7 @@ struct RsGenericSerializer : RsSerialType
/** Allow shared allocator usage to avoid costly JSON deepcopy for
* nested RsSerializable */
SerializeContext(
uint8_t *data, uint32_t size,
uint8_t* data = nullptr, uint32_t size = 0,
SerializationFlags flags = SERIALIZATION_FLAG_NONE,
RsJson::AllocatorType* allocator = nullptr) :
mData(data), mSize(size), mOffset(0), mOk(true), mFlags(flags),
@ -260,6 +260,12 @@ struct RsGenericSerializer : RsSerialType
static const SerializationFlags SERIALIZATION_FLAG_SIGNATURE; // 0x0002
static const SerializationFlags SERIALIZATION_FLAG_SKIP_HEADER; // 0x0004
/** Used for JSON deserialization in JSON API, it causes the deserialization
* to continue even if some field is missing (or incorrect), this way the
* API is more user friendly as some methods need just part of the structs
* they take as parameters. */
static const SerializationFlags SERIALIZATION_FLAG_YIELDING; // 0x0008
/**
* The following functions overload RsSerialType.
* They *should not* need to be further overloaded.

11
libretroshare/src/serialiser/rstypeserializer.cc
View File

@ -42,7 +42,8 @@
//static const uint32_t MAX_SERIALIZED_ARRAY_SIZE = 500 ;
static const uint32_t MAX_SERIALIZED_CHUNK_SIZE = 10*1024*1024 ; // 10 MB.
#define SAFE_GET_JSON_V() \
#ifdef RSSERIAL_DEBUG
# define SAFE_GET_JSON_V() \
const char* mName = memberName.c_str(); \
bool ret = jDoc.HasMember(mName); \
if(!ret) \
@ -50,10 +51,16 @@ static const uint32_t MAX_SERIALIZED_CHUNK_SIZE = 10*1024*1024 ; // 10 MB.
std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName \
<< "\" not found in JSON:" << std::endl \
<< jDoc << std::endl << std::endl; \
print_stacktrace(); \
return false; \
} \
rapidjson::Value& v = jDoc[mName]
#else // ifdef RSSERIAL_DEBUG
# define SAFE_GET_JSON_V() \
const char* mName = memberName.c_str(); \
bool ret = jDoc.HasMember(mName); \
if(!ret) return false; \
rapidjson::Value& v = jDoc[mName]
#endif // ifdef RSSERIAL_DEBUG
//============================================================================//

86
libretroshare/src/serialiser/rstypeserializer.h
View File

@ -57,9 +57,7 @@
{ \
/* Use same allocator to avoid deep copy */\
RsGenericSerializer::SerializeContext elCtx(\
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,\
RsGenericSerializer::SERIALIZATION_FLAG_NONE,\
&allocator );\
nullptr, 0, ctx.mFlags, &allocator );\
\
/* If el is const the default serial_process template is matched */ \
/* also when specialization is necessary so the compilation break */ \
@ -85,7 +83,7 @@
{\
using namespace rapidjson;\
\
bool& ok(ctx.mOk);\
bool ok = ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;\
Document& jDoc(ctx.mJson);\
Document::AllocatorType& allocator = jDoc.GetAllocator();\
\
@ -100,19 +98,21 @@
for (auto&& arrEl : jDoc[arrKey].GetArray())\
{\
RsGenericSerializer::SerializeContext elCtx(\
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,\
RsGenericSerializer::SERIALIZATION_FLAG_NONE,\
&allocator );\
nullptr, 0, ctx.mFlags, &allocator );\
elCtx.mJson.AddMember(arrKey, arrEl, allocator);\
\
T el;\
serial_process(j, elCtx, el, memberName); \
ok = ok && elCtx.mOk;\
\
ctx.mOk &= ok;\
if(ok) v.INSERT_FUN(el);\
else break;\
}\
}\
\
ctx.mOk = false;\
\
} while(false)
std::ostream &operator<<(std::ostream &out, const RsJson &jDoc);
@ -133,8 +133,8 @@ struct RsTypeSerializer
template<typename T>
typename std::enable_if<std::is_same<RsTlvItem,T>::value || !(std::is_base_of<RsSerializable,T>::value || std::is_enum<T>::value || std::is_base_of<RsTlvItem,T>::value)>::type
static /*void*/ serial_process( RsGenericSerializer::SerializeJob j,
RsGenericSerializer::SerializeContext& ctx,
T& member, const std::string& member_name)
RsGenericSerializer::SerializeContext& ctx,
T& member, const std::string& member_name )
{
switch(j)
{
@ -156,7 +156,8 @@ struct RsTypeSerializer
ctx.mOk = ctx.mOk && to_JSON(member_name, member, ctx.mJson);
break;
case RsGenericSerializer::FROM_JSON:
ctx.mOk = ctx.mOk && from_JSON(member_name, member, ctx.mJson);
ctx.mOk &= (ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
&& from_JSON(member_name, member, ctx.mJson);
break;
default:
std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
@ -194,8 +195,9 @@ struct RsTypeSerializer
to_JSON(member_name, type_id, member, ctx.mJson);
break;
case RsGenericSerializer::FROM_JSON:
ctx.mOk = ctx.mOk &&
from_JSON(member_name, type_id, member, ctx.mJson);
ctx.mOk &=
(ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
&& from_JSON(member_name, type_id, member, ctx.mJson);
break;
default:
std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
@ -287,15 +289,11 @@ struct RsTypeSerializer
{
// Use same allocator to avoid deep copy
RsGenericSerializer::SerializeContext kCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE,
&allocator );
nullptr, 0, ctx.mFlags, &allocator );
serial_process<T>(j, kCtx, const_cast<T&>(kv.first), "key");
RsGenericSerializer::SerializeContext vCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE,
&allocator );
nullptr, 0, ctx.mFlags, &allocator );
serial_process<U>(j, vCtx, const_cast<U&>(kv.second), "value");
if(kCtx.mOk && vCtx.mOk)
@ -316,7 +314,7 @@ struct RsTypeSerializer
{
using namespace rapidjson;
bool& ok(ctx.mOk);
bool ok = ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;
Document& jDoc(ctx.mJson);
Document::AllocatorType& allocator = jDoc.GetAllocator();
@ -336,9 +334,7 @@ struct RsTypeSerializer
if (!ok) break;
RsGenericSerializer::SerializeContext kCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE,
&allocator );
nullptr, 0, ctx.mFlags, &allocator );
if(ok)
kCtx.mJson.AddMember("key", kvEl["key"], allocator);
@ -346,9 +342,7 @@ struct RsTypeSerializer
ok = ok && (serial_process(j, kCtx, key, "key"), kCtx.mOk);
RsGenericSerializer::SerializeContext vCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE,
&allocator );
nullptr, 0, ctx.mFlags, &allocator );
if(ok)
vCtx.mJson.AddMember("value", kvEl["value"], allocator);
@ -356,14 +350,20 @@ struct RsTypeSerializer
ok = ok && ( serial_process(j, vCtx, value, "value"),
vCtx.mOk );
ctx.mOk &= ok;
if(ok) v.insert(std::pair<T,U>(key,value));
else break;
}
}
ctx.mOk = false;
break;
}
default: break;
default:
std::cerr << __PRETTY_FUNCTION__ << " Unknown serial job: "
<< static_cast<std::underlying_type<decltype(j)>::type>(j)
<< std::endl;
exit(EINVAL);
}
}
@ -566,7 +566,9 @@ struct RsTypeSerializer
case RsGenericSerializer::FROM_JSON:
{
uint32_t f;
ctx.mOk = from_JSON(memberName, f, ctx.mJson);
ctx.mOk &=
(ctx.mOk || ctx.mFlags & RsGenericSerializer::SERIALIZATION_FLAG_YIELDING)
&& from_JSON(memberName, f, ctx.mJson);
v = t_RsFlags32<N>(f);
break;
}
@ -623,9 +625,7 @@ struct RsTypeSerializer
// Reuse allocator to avoid deep copy later
RsGenericSerializer::SerializeContext lCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE,
&allocator );
nullptr, 0, ctx.mFlags, &allocator );
member.serial_process(j, lCtx);
@ -642,28 +642,32 @@ struct RsTypeSerializer
}
case RsGenericSerializer::FROM_JSON:
{
rapidjson::Document& jDoc(ctx.mJson);
RsJson& jDoc(ctx.mJson);
const char* mName = memberName.c_str();
ctx.mOk = ctx.mOk && jDoc.HasMember(mName);
bool hasMember = jDoc.HasMember(mName);
bool yielding = ctx.mFlags &
RsGenericSerializer::SERIALIZATION_FLAG_YIELDING;
if(!ctx.mOk)
if(!hasMember)
{
std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName
<< "\" not found in JSON:" << std::endl
<< jDoc << std::endl << std::endl;
print_stacktrace();
if(!yielding)
{
std::cerr << __PRETTY_FUNCTION__ << " \"" << memberName
<< "\" not found in JSON:" << std::endl
<< jDoc << std::endl << std::endl;
print_stacktrace();
}
ctx.mOk = false;
break;
}
rapidjson::Value& v = jDoc[mName];
RsGenericSerializer::SerializeContext lCtx(
nullptr, 0, RsGenericSerializer::FORMAT_BINARY,
RsGenericSerializer::SERIALIZATION_FLAG_NONE );
RsGenericSerializer::SerializeContext lCtx(nullptr, 0, ctx.mFlags);
lCtx.mJson.SetObject() = v; // Beware of move semantic!!
member.serial_process(j, lCtx);
ctx.mOk = ctx.mOk && lCtx.mOk;
ctx.mOk &= lCtx.mOk;
break;
}

12
libretroshare/src/use_libretroshare.pri
View File

@ -17,6 +17,18 @@ sLibs =
mLibs = $$RS_SQL_LIB ssl crypto $$RS_THREAD_LIB $$RS_UPNP_LIB
dLibs =
rs_jsonapi {
RS_SRC_PATH=$$system_path($$clean_path($${PWD}/../../))
RS_BUILD_PATH=$$system_path($$clean_path($${OUT_PWD}/../../))
RESTBED_SRC_PATH=$$system_path($$clean_path($${RS_SRC_PATH}/supportlibs/restbed))
RESTBED_BUILD_PATH=$$system_path($$clean_path($${RS_BUILD_PATH}/supportlibs/restbed))
INCLUDEPATH *= $$system_path($$clean_path($${RESTBED_BUILD_PATH}/include/))
QMAKE_LIBDIR *= $$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/))
# Using sLibs would fail as librestbed.a is generated at compile-time
LIBS *= -L$$system_path($$clean_path($${RESTBED_BUILD_PATH}/library/)) -lrestbed
}
linux-* {
mLibs += dl
}

11
retroshare-gui/src/gui/settings/WebuiPage.cpp
View File

@ -22,6 +22,9 @@ resource_api::ApiServerMHD* WebuiPage::apiServerMHD = 0;
resource_api::ApiServerLocal* WebuiPage::apiServerLocal = 0;
#endif
resource_api::RsControlModule* WebuiPage::controlModule = 0;
#ifdef RS_JSONAPI
JsonApiServer* WebuiPage::jsonApiServer = nullptr;
#endif
WebuiPage::WebuiPage(QWidget */*parent*/, Qt::WindowFlags /*flags*/)
{
@ -105,6 +108,10 @@ QString WebuiPage::helpText() const
// TODO: LIBRESAPI_LOCAL_SERVER Move in appropriate place
#ifdef LIBRESAPI_LOCAL_SERVER
apiServerLocal = new resource_api::ApiServerLocal(apiServer, resource_api::ApiServerLocal::serverPath());
#endif
#ifdef RS_JSONAPI
jsonApiServer = new JsonApiServer(9092);
jsonApiServer->start("WebuiPage::jsonApiServer");
#endif
return ok;
}
@ -126,6 +133,10 @@ QString WebuiPage::helpText() const
delete controlModule;
controlModule = 0;
}
#ifdef RS_JSONAPI
delete jsonApiServer;
jsonApiServer = nullptr;
#endif
}
/*static*/ void WebuiPage::showWebui()

7
retroshare-gui/src/gui/settings/WebuiPage.h
View File

@ -3,6 +3,10 @@
#include <retroshare-gui/configpage.h>
#include "ui_WebuiPage.h"
#ifdef RS_JSONAPI
# include "jsonapi/jsonapi.h"
#endif
namespace resource_api{
class ApiServer;
class ApiServerMHD;
@ -55,4 +59,7 @@ private:
static resource_api::ApiServerLocal* apiServerLocal;
#endif
static resource_api::RsControlModule* controlModule;
#ifdef RS_JSONAPI
static JsonApiServer* jsonApiServer;
#endif
};

19
retroshare.pri
View File

@ -115,6 +115,12 @@ rs_macos10.9:CONFIG -= rs_macos10.11
rs_macos10.10:CONFIG -= rs_macos10.11
rs_macos10.12:CONFIG -= rs_macos10.11
# To enable JSON API append the following assignation to qmake command line
# "CONFIG+=rs_jsonapi"
CONFIG *= no_rs_jsonapi
rs_jsonapi:CONFIG -= no_rs_jsonapi
###########################################################################################################################################################
#
# V07_NON_BACKWARD_COMPATIBLE_CHANGE_001:
@ -157,7 +163,7 @@ rs_v07_changes {
}
################################################################################
## RetroShare qmake functions goes here as all the rest may use them ###########
## RetroShare qmake functions goes here as all the rest may use them. ##########
################################################################################
## Qt versions older the 5 are not supported anymore, check if the user is
@ -273,7 +279,12 @@ no_sqlcipher {
rs_autologin {
DEFINES *= RS_AUTOLOGIN
warning("You have enabled RetroShare auto-login, this is discouraged. The usage of auto-login on some linux distributions may allow someone having access to your session to steal the SSL keys of your node location and therefore compromise your security")
RS_AUTOLOGIN_WARNING_MSG = \
You have enabled RetroShare auto-login, this is discouraged. The usage \
of auto-login on some linux distributions may allow someone having \
access to your session to steal the SSL keys of your node location and \
therefore compromise your security
warning("$${RS_AUTOLOGIN_WARNING_MSG}")
}
rs_onlyhiddennode {
@ -313,6 +324,10 @@ rs_chatserver {
DEFINES *= RS_CHATSERVER
}
rs_jsonapi {
DEFINES *= RS_JSONAPI
}
debug {
QMAKE_CXXFLAGS -= -O2 -fomit-frame-pointer
QMAKE_CFLAGS -= -O2 -fomit-frame-pointer

1
supportlibs/restbed Submodule

@ -0,0 +1 @@
Subproject commit c27c6726d28c42e2e1b7537ba63eeb23e944789d