monero-ccs-proposals/Modularize Monero

---
layout: fr
title:
author: Thor a.k.a vthor a.k.a DiosDelRayo
date: Augost 27, 2024
amount: 135
milestones:
  - name: First month
    funds: 45 XMR
    done:
    status: unfinished
  - name: Second month
    funds: 45 XMR
    done:
    status: unfinished
  - name: Third month
    funds: 45 XMR
    done:
    status: unfinished
payouts:
  - date:
    amount: 45 XMR
  - date:
    amount: 45 XMR
  - date:
    amount: 45 XMR
---
## About
This proposal aims to modularize the Monero codebase, transforming it from
it's current monolithic structure into a more modular codebase which can be
cherry-picked and taken apart for different use cases.

### Objectives:
(Ordered by importance)
1. 100% Backward compatibility with the exiting code case. The source base
will be transformed that a complete compile will yield still the same
outcome and functionality. Where necessary there will be created wrapper
code to stay compatible.
2. Separations of concerns. All functionalities should be separated modules
to cherry pick what to use and/or compile. E.g. if you build an hardware
wallet, you don't need a huge part of the code, but at the moment it is a
very difficult part to separate it. Maybe there even possibilities to run
on micro controller an hardware wallet, what at the moment looks already
out of scope. XmrSigner on a pi zero armv6@800MHz and 512MB RAM, needs >3
minutes to load or restore a wallet. But there is hope with the findings on
separating all in modules, to find better ways to do signing and handling
of outputs and key images.
3. Minimize external dependencies, all cryptographic code should be
included in the source code. External dependencies often lead to breaking
things later, maybe the domain vanished, or the author wants to make a
statement and takes it offline, etc. Cryptographic code anyway almost never
changes after established once, and normally the license allows to include
the that source code into your own. To make sure that changes because of
vulnerabilities/fixes get updated asap the source should be monitored if
possible. Other dependencies like boost should be minimized as far as
possible, and if not possible to get rid of the dependency at least it
should be possible to compile without if the provided functionality is not
needed. All code that can be substituted to Qt6/C++ will be substituted
with the hope of eliminate things like epee.
4. Improved readability and understandability. How almost all touching real
world value, a new developer should have it easier to understand what is
going on, then browsing through a source file with 15K LOC and then jump
through external helper functions or complicated constructs. Source should
be as simple as possible but not simpler.
5. Flexible compilation. The source should be restructured in a way that
you can cherry-pick which code to import into your project, have it easy to
update modules when modules change. Also should it be easy to generate a
library where you cherry-pick the needed modules/functionality.
6. Modular architecture. Redesigning the architecture so that it is
possible to exchange modules easily, like want to test a new blockchain
back-end, simply inherit from the blockchain interface and write the new
module using another database or protocol to retrieve the new blocks....
7. Comprehensive documentation. Every developer loves to have up to date
and complete documentation, and every developer hates to write it. In the
transformation the current monolithic source and the new modular source
should get a top notch documentation, because before transforming the
source one anyway first needs to understand, and so 70% of the
documentation work is almost done, only missing to write it down and phrase
it in the best possible way. After transforming also the differences and
what the code does is pretty clear and therefore is the best moment in time
to write. This effort should benefit the current like the new
implementation equal.
8. Enhanced auditability. Through a more broken down and easier to read
code base and improved structure and documentation, both current and new
source should be easier to audit by entities not familiar with the source
yet. (If you audit source you already know, you get also blind to some
issues, because they are simply familiar).
9. Facilitate future translations. The modularized structure with clear
interfaces and comprehensive documentation should it end also lead to
potential future translations into other languages like Rust, Kotlin,
Erlang or Python (the current monero-python library is e.g. almost only a
wrapper for the wallet rpc).
10. Cleaner Code. Somehow a bit redundant to point 2, 3 and 4, only
addition to adhere to modern C++ practices and clean code principles.

### Non-objectives:
1. Performance. Of course the code should be as performed as possible, but
no objective will be sacrificed for performance, clarity above performance.
2. Substitute the work of the original authors or diminish the worth of
their work. Their work is highly respected and cooperation is searched to
tame the monster. It is only natural that the monster grows as long you
feed it. And it is logical that you feed the monster if you love it and if
it lives in your house. That this source has become a so monolithic beast
is almost unavoidable. Of course it would be cool if one day this effort
will lead to the core development on the more modularized code base, and
for that I will search also current developers opinions while walking the
path, but the transformation is not done for that purpose. The current
developers have light-years more experience and knowledge in that source -
but like normal (I assume) no time to refactor the whole monster - even
less while worked on that very source actively.
3. Another language, introducing new libraries or tool-kits.  The very
purpose is less of all with the same functionality. And sticking with
Qt6/C++ is very intentional, first needs to be a modularization and
comprehensive documentation, only after that it can be thought to go a step
further IMO.

### Challenges
1. Transforming source which is under current development. Somehow like
making a surgery on a person out for a walk, while walking... But I hope I
can mitigate this challenge in working on modules, one at the time and have
like a recipe from A to B, so if there are not huge changes it should not
derail me.
2. Qt6/C++ newest C++ standards and best practices how I'm a novice on C++.
I develop for more then 20 years now, but touched during that 20 years only
C and once for a hack I modified C++ code and only recently I started with
Qt/C++. So as usual I need to get up to speed while I'm doing. It is always
easier (in the beginning) to modify source in a language you are not fit
yet then write from ground up until it flips and writing from ground up
will be easier. But I still in the phase where I will have to read some
books in parallel.
3. The time effort is impossible to estimate as long the the source is not
completely understood, and even then it is hard - but I assume as soon the
complete source is understood, almost all the work is also done.

## Who
Me, Thor a.k.a vThor a.k.a DiosDelRayo, I'm about to finishing the
XmrSigner [Monero Signer
Resurrection](https://ccs.getmonero.org/proposals/%20MoneroSignerResurrecti
on.html) and encountered various difficulties on getting XmrSigner
production ready, like main points monero-python is almost only a mere
wrapper for wallet RPC, the monero source is (almost) take all and digest
or take nothing. So finishing this proposal will be directly beneficial for
XmrSigner and probably even lead to a XmrSigner NG dropping a lot of
ballast, going directly bare metal and not using an interpreted language
like Python, but Qt/C++, Rust or Zig. Since more then 20 years I'm in the
IT field, from developing (where I started) over network administration, to
secure communication systems. Wrote code in a lot of different languages
and prefer always to learn what is needed on the way instead of only making
that what I know fitting for the operation. The hammer I know well, but it
is not always the best tool to archive anything.

## Why it is important for community
- It would make developing new wallets or even projects not heard about
easier for new developer and so attracting more developers creating
products for the Monero Ecosystem.
- It would improve audits of the monero source.
- Me getting familiar with the monero source would create also the
opportunity to help one day on that very source.
- Possibility of finding hidden bugs, how I will need to read and
understand the source code intensively.
- Maybe the possibility to get parts of monero working on even more
restricted/limited hardware like micro controller for hardware wallets or
similar products
- Getting XmrSigner well done, at the moment the state is unsatisfying at
best.

## Milestones
How it is impossible to predict or estimate anything as long not familiar
with the whole source, I prefer to propose to work at least 130h/month for
45 XMR (don't care about the exchange rate because XMR should be the
anchor) for 3 months, and very probably writing a new proposal to continue.
All what I work above 130h in one month shall go into the next
month/milestone, I will give a biweekly report below the proposal in GitLab
and commit and push the changes each day (normally I push only after a big
junk is finished, but here it makes sense to push daily for accountability
reasons).

Results in 3 Milestones of 130 hours for 45 XMR each, in total 390 hours
for 135 XMR.

## Project Timeline
Project timeline is open how time cannot really be estimated (at least for
now, maybe it becomes easier after the third milestone). So it is simply to
process as much as is possible per day.