-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Support for large file import, display upload progress #1812
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gpapp
wants to merge
44
commits into
docmost:main
Choose a base branch
from
gpapp:Support-for-large-file-import,-display-upload-progress
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,007
−95
Open
Changes from all commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
995536e
Added functionality to allow share to Docmost PWA from Android
gpapp 35b62c8
Added automatic version refresh
gpapp 2a9a27d
Added share to a parent page as an option.
gpapp 821fdd5
Fixed title line duplication
gpapp 97ce787
Added simple page selector
gpapp c4ec8e4
Fixed title stripping
gpapp 0ae57c3
Streamlined sharing logic to handle title/
gpapp cac2993
Fixed page name in share target
gpapp 17591b2
Added replace=true, to avoid history poisoning
gpapp b5c6101
Moving content generation logic
gpapp d0aa2c5
Trimming content's first line if it matches the title
gpapp ff57530
Fixing manifest.json icon sizes
gpapp 95c3238
Remembering last used workspace/page
gpapp 717da91
Fixed event listener
gpapp 0b18676
Sanitized input
gpapp 3a58100
Added optional title parameter, not to rely on sanitized filename.
gpapp ad893bc
Added parentPageId validation
gpapp c0152c6
Coderabbit recommended sanity fixes
gpapp 7ca8054
Fixed access rights for imports. New pages should require CRETE permisss
gpapp f162687
Resolved double import issue. Added search whitespace handling
gpapp c7a0781
Support progress feedback
gpapp 7d8c5c6
Allow upload of large files
gpapp b8aa94c
Added postgres healthcheck to docker, .env variables to DB setup.
gpapp e0f8981
Display import progress
gpapp d5e2ddd
Allow import of invalid URLs, but adding logging.
gpapp 3ee4493
Missed commit
gpapp 135025c
Speed up build using cached pnpm
gpapp e2955b9
Fixed copying patches
gpapp 6740e29
Handling malformed URLs gracefully
gpapp ecaa6a1
Adding typescript to editor-ext
gpapp 91d69bb
Increased concurrency
gpapp 4b2439e
Added frozen typescript version
gpapp c0a330b
Added editor dependencies
gpapp bb0bd73
Rolling back builder changes
gpapp 529d7dc
Fixing build
gpapp d4e11c2
Build order fix
gpapp 609dffe
Simplified Dockerfile ownership
gpapp 6b949a1
Reduce build frequency
gpapp 4b408e7
Reduced copy of files in Dockerfile
gpapp 6646095
Fixing permission issue
gpapp b8c7d2d
Update docker-compose.yml
gpapp 31716a0
feat: Implement attachment service for file and image uploads, suppor…
gpapp f7f3158
Merge branch 'main' into Support-for-large-file-import,-display-uploa…
gpapp a340a9a
Merge branch 'docmost:main' into Support-for-large-file-import,-displ…
gpapp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,24 @@ | ||
| FROM node:22-slim AS base | ||
| LABEL org.opencontainers.image.source="https://github.com/docmost/docmost" | ||
| ENV PNPM_HOME="/pnpm" | ||
| ENV PATH="$PNPM_HOME:$PATH" | ||
| RUN corepack enable && corepack prepare [email protected] --activate | ||
| WORKDIR /app | ||
|
|
||
| RUN npm install -g [email protected] | ||
|
|
||
| FROM base AS builder | ||
|
|
||
| WORKDIR /app | ||
| COPY .npmrc package.json pnpm*.yaml nx.json /app/ | ||
| COPY apps/client/package.json /app/apps/client/package.json | ||
| COPY apps/server/package.json /app/apps/server/package.json | ||
| COPY packages/editor-ext/package.json /app/packages/editor-ext/package.json | ||
|
|
||
| COPY . . | ||
| COPY patches /app/patches | ||
|
|
||
| RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile | ||
|
|
||
| RUN pnpm install --frozen-lockfile | ||
| COPY . . | ||
| RUN pnpm build | ||
|
|
||
| FROM base AS installer | ||
|
|
@@ -18,32 +27,27 @@ RUN apt-get update \ | |
| && apt-get install -y --no-install-recommends curl bash \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Copy apps | ||
| COPY --from=builder /app/apps/server/dist /app/apps/server/dist | ||
| COPY --from=builder /app/apps/client/dist /app/apps/client/dist | ||
| COPY --from=builder /app/apps/server/package.json /app/apps/server/package.json | ||
|
|
||
| # Copy packages | ||
| COPY --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist | ||
| COPY --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json | ||
| RUN mkdir -p /app/data/storage && chown -R node:node /app | ||
|
|
||
| # Copy root package files | ||
| COPY --from=builder /app/package.json /app/package.json | ||
| COPY --from=builder /app/pnpm*.yaml /app/ | ||
| COPY --from=builder /app/.npmrc /app/.npmrc | ||
| COPY --chown=node:node --from=builder /app/package.json /app/package.json | ||
| COPY --chown=node:node --from=builder /app/pnpm*.yaml /app/ | ||
|
|
||
| # Copy patches | ||
| COPY --from=builder /app/patches /app/patches | ||
| COPY --chown=node:node --from=builder /app/patches /app/patches | ||
|
|
||
| RUN chown -R node:node /app | ||
| # Copy apps | ||
| COPY --chown=node:node --from=builder /app/apps/server/dist /app/apps/server/dist | ||
| COPY --chown=node:node --from=builder /app/apps/client/dist /app/apps/client/dist | ||
| COPY --chown=node:node --from=builder /app/apps/server/package.json /app/apps/server/package.json | ||
|
|
||
| USER node | ||
| # Copy packages | ||
| COPY --chown=node:node --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist | ||
| COPY --chown=node:node --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json | ||
|
|
||
| RUN pnpm install --frozen-lockfile --prod | ||
| USER node | ||
|
|
||
| RUN mkdir -p /app/data/storage | ||
| RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile --prod | ||
|
|
||
| VOLUME ["/app/data/storage"] | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| const VERSION = 'v4'; // Update this to trigger re-install | ||
|
|
||
| self.addEventListener('install', (event) => { | ||
| event.waitUntil(self.skipWaiting()); | ||
| }); | ||
|
|
||
| self.addEventListener('activate', (event) => { | ||
| event.waitUntil(clients.claim()); | ||
| }); | ||
|
|
||
| self.addEventListener('fetch', (event) => { | ||
| const url = new URL(event.request.url); | ||
|
|
||
| if (event.request.method === 'POST' && url.pathname === '/share-target') { | ||
| event.respondWith( | ||
| (async () => { | ||
| let data = { title: "", text: "", url: "" }; | ||
| try { | ||
| const formData = await event.request.formData(); | ||
| for (const [key, value] of formData.entries()) { | ||
| if (key in data && typeof value === "string") data[key] = value; | ||
| } | ||
| } catch (e) { | ||
| // If parsing fails, still redirect to UI (which can show "no content"). | ||
| } | ||
|
|
||
| // Store in Cache API | ||
| const cache = await caches.open('share-target'); | ||
| await cache.put( | ||
| 'shared-content', | ||
| new Response(JSON.stringify(data), { | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| }) | ||
| ); | ||
|
|
||
| // Redirect to the client-side route | ||
| return Response.redirect('/share-target', 303); | ||
| })() | ||
| ); | ||
| } | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: docmost/docmost
Length of output: 2566
🏁 Script executed:
# Check the client-side type definition cat apps/client/src/features/file-task/types/file-task.types.tsRepository: docmost/docmost
Length of output: 432
🏁 Script executed:
Repository: docmost/docmost
Length of output: 86
🏁 Script executed:
Repository: docmost/docmost
Length of output: 86
🏁 Script executed:
Repository: docmost/docmost
Length of output: 86
🏁 Script executed:
Repository: docmost/docmost
Length of output: 671
🏁 Script executed:
Repository: docmost/docmost
Length of output: 782
🏁 Script executed:
Repository: docmost/docmost
Length of output: 15046
🏁 Script executed:
Repository: docmost/docmost
Length of output: 41
Update type to allow null values from server.
The server-side FileTasks type defines
progress: Generated<number | null>(db.d.ts:172), which can return null values. The client-side IFileTask type should reflect this asprogress?: number | nullinstead ofprogress?: number. While the code at page-import-modal.tsx:204 already handles this case withconst progress = fileTask.progress || 0, the type definition should be accurate to prevent type-checking issues in other code paths that consume this interface.🤖 Prompt for AI Agents