This adds parsing of page content so that headers apply a boost to
scores in the search term index.
Additionally, this merges title and content terms to reduce the amount
of stored terms a little.
Includes testing to cover.
Checked over latest changes for potential SQL injection, all variable
usages are either (from trusted sourced AND case) or using
parameters/bindings to ensure it's handled at driver/lib level.
Relations now loaded during back-end query phase instead of being lazy
loaded one-by-one within views.
Reduced queries in testing from ~60 to ~20.
Need to check other areas list-item.php's "showPath" option is used to
ensure relations are properly loaded for those listings.
Adds adjustment of search term 'score' (Using in result ranking) so that
a relative 0.3 to 1.3 mulitplier is applied based upon relative
popularity within the whole database. At this point the term popularity
is still done via a prefix match against the search term.
Uses a SUM(IF(cond, a, IF(cond, a, ...))) chain to produce the scoring
result in the select query.
- Moved out extension check to ImageService as that seems more relevant.
- Updated models to use static-style references instead of facade to align with common modern usage within the app.
- Updated custom image_extension validation rule to use shared logic in image service.
Since AzureAD graph is going away.
Tested using old AzureAD graph usage for backwards-compatbility, did not
seem to break things. Could not test with conditional access though due
to azure never enforcing it no matter what I attempted.
Fpr #3028
- Migrated env usages to config.
- Removed potentially unneeded config options or auto-set signed options
based upon provision of certificate.
- Aligned SP certificate env option naming with similar IDP option.
Tested via AFDS on windows server 2019. To test on other providers.
Session was being lost due to the callback POST request cookies
not being provided due to samesite=lax. This instead adds an additional
hop in the flow to route the request via a GET request so the session is
retained. SAML POST data is stored encrypted in cache via a unique ID
then pulled out straight afterwards, and restored into POST for the SAML
toolkit to validate.
Updated testing to cover.
Provides a simple bookstack focused view that does not rely on JavaScript.
Contains links to BookStack specific resources in addition to commonly
desired debug details.
- Files within the storage/ path could be accessed via path traversal
references in content, accessed upon HTML export.
- This addresses this via two layers:
- Scoped local flysystem filesystems down to the specific image &
file folders since flysystem has built-in checking against the
escaping of the root folder.
- Added path normalization before enforcement of uploads/{images,file}
prefix to prevent traversal at a path level.
Thanks to @Haxatron via huntr.dev for discovery and reporting.
Ref: https://huntr.dev/bounties/ac268a17-72b5-446f-a09a-9945ef58607a/
- Prevents authenticated responses being visible when back button
pressed in browser.
- Previously, 'no-cache, private' was added by default by Symfony which
would have prevents proxy cache issues but this adds no-store and a
max-age option to also invalidate all caching.
Thanks to @haxatron via huntr.dev
Ref: https://huntr.dev/bounties/6cda9df9-4987-4e1c-b48f-855b6901ef53/
- Removed uneeded custom refresh or logout actions for OIDC.
- Restructured how the services and guards are setup for external auth
systems. SAML2 and OIDC now directly share a lot more logic.
- Renamed any OpenId references to OIDC or OpenIdConnect
- Removed non-required CSRF excemption for OIDC
Not tested, Come to roadblock due to lack of PHP8 support in upstream
dependancies. Certificate was deemed to be non-valid on every test
attempt due to changes in PHP8.
- Made oidc config more generic to not be overly reliant on the library
based upon learnings from saml2 auth.
- Removed any settings that are redundant or not deemed required for
initial implementation.
- Reduced some methods down where not needed.
- Renamed OpenID to OIDC
- Updated .env.example.complete to align with all options and their
defaults
Related to #2169
- Added testing to cover warning cases.
- Refactored logic to be simpler and move much of the business out of
the controller.
- Added new message that's more suitable to the case this was handling.
- For detecting an outdated draft, checked the draft created_at time
instead of updated_at to better fit the scenario being checked.
- Updated some method types to align with those potentially being used
in the logic of the code.
- Added a cache of shown messages on the front-end to prevent them
re-showing on every save during the session, even if dismissed.
Review of #2935
- Removed from .env files and added warnings for use if found in config
file.
- Updated permission service to use whereColumn queries to auto-handle
use of prefixes.
Review of #2936
- Added testing to cover
- Added APP_PROXIES to .env.example.complete with details.
- Renamed migration to better align the name and to set the migration
date to fit with production deploy order.
- Removed index from IP column in migration since an index does not yet
provide any value.
- Updated table header text label.
- Prevented IP recording when in demo mode.
Done a little reorganisation while there of misplaced tests.
Moved MarkdownTest to a new PageEditorTest to avoid confusion with
other markdown elements and to align with other page tests.
Was causing things like emjoi within script content to be somewhat
mangled. Instead we force UTF8 only parsing via XML declaration.
Added test to cover.
For #2923
Fixes a strange issue of HTML tags within script tags being malformed
when part of the HTML custom head content due to the PHP parsing we do.
DOMDocument seemed to cause this upon load.
Adding LIBXML_SCHEMA_CREATE to the ->loadHTML call seems to fix this but
not really sure why. Doesn't seem to cause further issues though.
Tested with multiple scripts and styles and comments and meta tags.
- Also added new testing class to cover.
- As part of testing, added new folder within tests to house setting
specific tests.
For #2914
In particular, That around the casing of dangerous values within
attributes. This uses some xpath translation to handle different casing
in contains searching.
Injected db instance was causing the DB connection to be
made a lot earlier than desired or required.
Swapped to a facade for now but ideally this extension of services needs
to be cleaned up with a better approach in general.
WKHTMLtoPDF provides limited control for external fetching
so that will now be disabled by default unless
ALLOW_UNTRUSTED_SERVER_FETCHING=true is specifically set.
This new option will also control DOMPDF fetching.
During writing of the update notes, found that the upgrade path would be
tricky from a security point of view. If people were pending email
confirmation but had an active session, they could technically be
actively logged in after the next release.
Added middlware as an extra precaution for now.
- Numbered and bullet list margins have been made consistent
- Numbered lists margins were increase at some point to handle 3-digit
numbers, Normal bullet margins updated to match this.
- Consistent margin for sub-lists.
- System back-end markdown renderer (For pages) updated with a custom
list item renderer to apply class for to align with front-end renderer.
- This means that task list items will be consistent with the preview
and not render a number/bullet.
- Indentation styles for task list items fixed to be visually indented.
For #2854 and #2837
- Primarily moved and re-organised view files.
- Included readme within views to document the convention.
- Fixed some issues with page field select list in previous commit.
- Tweaked some route names while going through.
- Split some views out further.
Closes#2805
- Loaded book_slug as part of chapter/page queries instead of books
being loaded in afterwards.
- Removed unused page method.
- Updated some page queries to load specific attributes.
- Updated auth system for mfa to not update intended URL so that the
user is not redirected to mfa setup after eventual login.
- Added notification for users setting up MFA, after setup when
redirected back to login screen to advise that MFA setup was complete
but they need to login again.
- Updated some bits of wording to display better.
- Restructured some of the route naming to be a little more consistent.
- Moved the routes about to be more logically in one place.
- Created a new middleware to handle the auth of people that should be
allowed access to mfa setup routes, since these could be used by
existing logged in users or by people needing to setup MFA on access.
- Added testing to cover MFA setup required flow.
- Added TTL and method tracking to session last-login tracking system.
Also added testing to cover as part of this in addition to adding the
core backup code handling required.
Also added the standardised translations for switching mfa mode and
adding testing for this switching.
Also continued a bit on the MFA verification system.
Moved some MFA routes to public space using updated login service to get
the current user that is either logged in or last attempted login (With
correct creds).
Discovered some difficult edge cases:
- User image loading in header bar when using local_secure storage
- 404s showing user-specific visible content due to content listing on
404 page since user is in semi-logged in state. Maybe need to go
through and change up how logins are handled to centralise and
provide us better control at login time to prevent any auth level.
- Added new DB column for control and role updated create/update actions.
- Created new middleware as a start to actual enforcement logic.
- Added indicator to role list of whether MFA is enforced.
System was using the empty state return from theme_path,
when no theme was configured, for loading in languages
which would result in the root path being looked up upon.
This changes the theme_path helper to return null in cases a theme
is not configured instead of empty string to help prevent assumed
return path will be legitimate, and to help enforce error case
handling.
For #2836
On restore, added a link to the parent deletion restore if any exists
on a cascading parent. Added a test to cover this case to ensure its shown.
Also tweaked default empty state message on recycle bin item list to align
with new column count.
Also done a little existing code cleanup including a getUrl helper on
the deletion items.
Related to #2682 & #2594
For review of meta tag additions as per PR #2393.
This commit removes any image guesswork and only uses images that have
been set by the author for the specific content.
This also adds tests to cover the expected OG tags.
- Removed ZIP system for now, until the idea can be fleshed out.
- Added testing to cover.
- Upgraded used library.
- Added custom handling for BookStack callouts.
- Added HTML cleanup to better produce output for things like code
blocks.
Fixes issues thrown in custom HMTL head & page content filtering when
the content is comprised of only a single HTML comment.
Adds tests to cover.
For #2804
A query string will cause attachments to be provided inline
with an appropriate mime type.
Remaining actions:
- Tests
- Front-end functionality
- Config option?
- Added test cases to cover.
- Altered parsing logic to be a little less reliant on regex.
- Added new iamge repo method for creating from data.
- Added extension validation and additional type support.
- Done some cleanup of common operations within PageContent.
- Added message to API docs/method to mention image usage.
For #2700 and #2631.
- Updated styling to include item name.
- Extracted used text to translations.
- Updated the design to better suit the surrounding blocks.
- Removed newly added model/repo methods.
- Moved core logic out of controller and instead into a "NextPreviousContentLocator"
helper with re-uses the output from the book-tree generation.
- Also added the system to chapters.
For #2511
- Reduced options to single new configuration paramter instead of two.
- Moved more logic into UserAvatars class.
- Updated LDAP avatar import to also run on login when no image is
currently set.
- Added thumbnail fetching to search requests.
- Added testing to cover.
Related to PR #2320, and issue #1161
- Refactored some tag code bits while reviewing.
- Updated tag design in search listing to be more subtle.
- Moved tags out of entity-list-item-basic template and instead moved
them into entity-list-item, below the existing content.
- Tweaked existing tag colors a little.
- Changed tag icon to be more tag-like.
- Added tag-on-search test case.
Review of #2487, Related to #2462
Allows someone using the theme system to configure the social driver
before a redirect action occurs, by passing a callback as an additional
param to the theme 'addSocialDriver' method.
To ensure thhat the visibility is still set on local storage options
since the previous recent changes could cause problems where in
scenarios where the server user could not read images uploaded by the
php process user.
Closes#2758
- Also removed some old view service references.
- Updated TopFavourites query to be based on favourites table and join
in the views instead of the other way around, so that favourites still
show even if they have no views.
- Removed old view system and started use of new query classes instead.
- Finished off RelationMultiModelQuery but found it was less efficient
than x-many queries due to the amount of tables being scanned.
Adding now for history but will delete as not used.
- Updated recently viewed to use same query system as popular items
rather than running and joining x-entities queries.
- Added "Most Viewed Faviourites" listing to homepages.
Phasing out the view service from being a generic 'service' class,
moving the core create/delete methods into the model.
The idea is that the existing query work will need to interlink
with the favourite system so maybe we have a (or many composable)
query building classes rather than mixing query building and
create/delete work as per the old service.
Also allowed config-centralised default user settings for this change
and bought existing user-level view options into that default settings
system to be cleaner in code usage.
For #2081
- Moved the ldap function out to our separate service for easier
testing.
- Added testing for the option.
- Moved tls_insecure part back up above connection start as found more
reliable there.
Done a lot of real-connection testing during this review.
Used wireshare to ensure TLS connection does take place.
Found LDAP_TLS_INSECURE=false can action unreliably, restarting php-fpm
helped.
Tested both trusted and untrusted certificates.
Also ensured revisions were not created more often than expected.
Summary field null check was triggering revision save even when empty
since it was still in request.
Related to #1846
This creates another mysql_testing database during db service setup
Replace server with env tags in phpunit.xml in order to force
override certain parameters when tests are run. See:
https://github.com/sebastianbergmann/phpunit/issues/2353 for more
information.
Rename primary developer Docker database from bookstack-test to
bookstack-dev. bookstack-test is used as the mysql_testing database
