electron prisma sqlite 2
[toc]
PASS: test putting prisma | @prisma/client
under release package.json
✅ cannot start when integrated with sqlite in packed version
suppose:
APP=release/app
step 1. add dependency in APP/package.json
// release/app/package.json
"dependencies": {
"sqlite3": "^5.0.2"
},
step 2. rebuild project
:::warn maybe we should node i -g node-gyp
first :::
[npm run rebuild] electron-rebuild --parallel --types prod,dev,optional --module-dir $APP
step 3. build and pack again
npm run build
[npm run app:dir] electron-builder --dir
ref:
chore(deps): upgrade node-gyp to ^6.0.1 by malept · Pull Request #327 · electron/electron-rebuild
sqlite - electron-packager with sqlite3 and webpack - Stack Overflow
step 4. PLUS: if we don't pack sqlite3
into $APP, then we won't pass the webpack build process only if we add this
// .erb/configs/webpack.config.base.ts
externals: [
...Object.keys(externals || {}),
// for prisma build
// https://github.com/prisma/prisma/issues/6564#issuecomment-899013495
{
_http_common: '_http_common',
},
// for sqlite3 build
// https://stackoverflow.com/a/51256562/9422455
{ sqlite3: 'commonjs sqlite3' },
],
✅ can normally open if no database connection in source code
// src/main/db.ts
interface FakePrisma {
erp: {
create: any;
upsert: any;
findMany: any;
};
}
export const prisma: FakePrisma = {
erp: {
create: () => {
console.log('fake-database: creating one');
},
findMany: () => {
console.log('fake-database: finding many');
},
upsert: () => {
console.log('fake-database: upserting one');
},
},
};
using pure sqlite3
and the programme goes well under prod, then failed with a popup suggesting me to integrate sqlite3
integrate prisma like sqlite
When add the "@prisma/client": "^3.7.0"
into APP/package.json
:
So we then try to add .prisma
into it.
"dependencies": {
"@prisma/client": "^3.7.0",
"sqlite3": "^5.0.2"
},
"devDependencies": {
"prisma": "^3.7.0",
"@prisma/client": "^3.7.0"
}
After I moved the libxxxxnode
into main
, then things are ok, except still too slow!
I try to debug as Other packages make Prisma load very slow · Issue #8484 · prisma/prisma did:
I found this app used my local packages, which maybe the issue behind the load problem.
So I uninstalled the global prisma:
npm uninstall -g prisma
Then I run again.
However, it starts to use the packages in my dev project.
Finally I do a copy before I run for test:
# $erb
# copy
cp prisma/schema.prisma $app/Contents/Resources/app/dist/main/
cp node_modules/.prisma/client/libquery_engine-darwin.dylib.node $app/Contents/Resources/app/dist/main
# run
open release/build/mac/皇家小虎快递分析系统.app/Contents/MacOS/皇家小虎快递分析系统
✨ maybe it's because prisma is searching file location, so that the system runs slow! (since finally it didn't search any schema.prisma
, which enlightened me!)
I am quite sure now the problem is about package import with prisma
I added one log just at the entrance of the whole programme.
And I am lucky to see it is printed after programme runs in binary, and then wait, which indicates the start of programme is ok, except the packages.
Yeah, thanks Mac's console, I saw the dawn ✨✨:sparkles
Stupid!
I found nothing helpful!
(Since the programme didn't cause any system exception, except block)
Finally found the problem, it's about the block of searching path
Just add these two lines for debug the findSync
sentence.
Then you would know where does it go wrong.
I located to the source code.
It seems the path generated incorrectly.
Since this directory is generated via cli, I re-generate it, which looks more normal then.
And then we pack again.
npm run build
npm run app:dir
Look into what's new in the APP/../main.js
OK, it's consistent with the source code after we prisma generate
.
However, there's no node_modules
neither .prisma
under main.js
directory now:
➜ main git:(main) ✗ ls
main.js main.js.LICENSE.txt preload.js
It's because we didn't pack it into app.asar
in electron.
So how to achieve this goal?
- we cannot use
externalResources
choice since it only pack files intoResources/
notapp/main/
- if we use
files
, then the files would list inapp.asar
underapp
directory, this is what we are looking for. - if we add the dependencies under
app/package.json
, then there would benode_modules
installed underapp
. It is also suitable if there would generate a.prisma/client
, we can give it a shot only if we add@prisma/client
intoapp/package.json
.
// release/app/package.json
"dependencies": {
"sqlite3": "^5.0.2",
"@prisma/client": "^3.7.0"
}
However, I tried but to find there is NO .prisma
folder in the target place.
And I tried locally to find that if I do not use prisma generate
then I only got an blank template index.js
, which is certainly not I want.
So, the answer is clear now: we can't put @prisma/client
into app/package.json
, since it won't generate the right sql model unless we use prisma generate
. And even the .prisma
generated locally, it won't be packed into out target application's app.asar folder.
Therefore, the only way best suitable for us is to manually copy the local .prisma
folder to app/dist/main/
so that can be searched by prisma
in the binary application.
We then use prisma generate
to synchronize the node_modules/.prisma/client/index.js
with out prisma/schema.prisma
file (which would copied into node_modules/.prisma/client/schema.prisma
), along with the generated query engine for MacOS: node_modules/.prisma/client/libquery_engine-darwin.dylib.node
.[TODO: what should windows user do; maybe copy all the files into app is not ideal for cross platform use. Instead, we should consider about only copy the schema.prisma
file, and let all the other work generated by platforms]
I finally understand why programme start by double click
would be so slow
I added log before findSync
again, and run by double click of the main programme.
And it should tell me that the process.cwd()
is /
that means from my computer root!
As a contrast, I ran it according the electron helper which is within the main programme.
And the process.cwd()
is ~
, which is much much faster!
what's the findSync
So, according the source code, the findSync
starts from the root
, then to search target pattern
.
Thus, from the commands before, we can see, it would search target pattern node_modules/.prisma/client
or .prisma/client
from process.cwd()
(i.e. /
when run main programme and ~
when run in shell)
What's the solution?
directly change the logic into
_ = __dirname
, and put.prisma/client
under the__dirname
, i.e.app/main
perhaps there's way (i.e. package or not) to not go this logic, which is so-called best practice? But sadly, I haven't found one after many times of hard search in github of prisma. I am looking forward the developers of prisma can give one.
- For example, maybe there is a way allows me to fill some parameters so that to disable the hard search or change the root from?
ref:
Current Solution: avoid `findSync'
avoid any call of findSync
// node_modules/.prisma/client/index.js
// to save the hard search time(maybe from '/'), let us directly point out the target prisma client place
console.log({process_cwd: process.cwd(), __dirname});
const dirname = path.join(__dirname, '.prisma/client');
console.log({dirname});
// const { findSync } = require('@prisma/client/runtime')
//
// const dirname = findSync(process.cwd(), [
// "node_modules/.prisma/client",
// ".prisma/client",
// ], ['d'], ['d'], 1)[0] || __dirname
// main files structure of an packed electron on Mac
// with `.prisma/client` besides `main.js`
// generate via ` find . ! -path '*node_modules/*' ! -path '*Frameworks/*' ! -path '*assets*' ! -path '*lproj' ! -path '*svg' | tree --fromfile --noreport -aC`
. // Main Entrance
└── Contents
├── Frameworks
├── Info.plist
├── MacOS
│ └── HJXH-DATA-ANALYSIS // Script Entrance
├── PkgInfo
└── Resources
├── app
│ ├── dist
│ │ ├── main
│ │ │ ├── .prisma
│ │ │ │ └── client
│ │ │ │ ├── index-browser.js
│ │ │ │ ├── index.js
│ │ │ │ ├── libquery_engine-darwin.dylib.node
│ │ │ │ ├── package.json
│ │ │ │ └── schema.prisma
│ │ │ ├── main.js // Main Runner
│ │ │ ├── main.js.LICENSE.txt
│ │ │ └── preload.js
│ │ └── renderer
│ │ ├── index.html
│ │ ├── renderer.js
│ │ └── renderer.js.LICENSE.txt
│ ├── node_modules
│ └── package.json
└── icon.icns
make binary can use prisma db push
The question is to intercept prisma
into package use, am I right?(maybe not...)
Anyway, we'd better have a try at the first: add prisma
as a dependency of app.
Now, we have prisma
in node_modules.
However, I doesn't be positive about running with prisma
since it's only a dependency not a command.
There are at least two kinds of commands we can use, one for global, one for those under node_modules/.bin
.
The sad truth is that only when we run npm i prisma
in packed app, we can have a prisma
under node_modules/.bin
, which seems unacceptable. [TODO: so why they cannot generate a prisma
under .bin
when I have assigned prisma
in package.json
of released app]
➜ app git:(main) ✗ npm i prisma
added 2 packages, and changed 1 package in 2s
3 packages are looking for funding
run `npm fund` for details
➜ app git:(main) ✗ ls node_modules/.bin
detect-libc node-gyp prisma rimraf sshpk-sign which
mkdirp node-pre-gyp prisma2 semver sshpk-verify
needle nopt rc sshpk-conv uuid
So let's have a try step by step.
First, only add prisma
in the package.json
of release.
若在release/package.json
里指定了@prisma/client
,则打包里没有.prisma/client/index.js
若未指定(只指定一个sqlite3
),则有
若直接指定两个(2022-01-06 02:15:12),则像一个一样还是报错了,但是我联想到了rebuild
So when I used rebuild
(in fact, it's npm run post-install
instead of npm run rebuild
), everything is okay now! (although with a lot of errors...)
rebuild
确实是一大解药,rebuild
完之后还要把prisma
移到对应的node_modules
目录旁,运行一遍prisma generate
,目前我一套流程走下来,将prisma
一步步同步,效果还行;只不过最后一步还是有问题,感觉双击程序启动找不到路径的问题还是存在。既然这样,步步同步的方式似乎也没啥性价比了
再测试一下步步同步的方式是否可以装 windows,如果也不行,我就直接放弃 prisma 了
确实不行,只有darwin
,没有看到给win
用的 query-engine.
在 binary 里需要用npm i prisma
安装脚本
➜ app git:(main) ✗ ls node_modules/.bin
detect-libc needle node-pre-gyp rc semver sshpk-sign uuid
mkdirp node-gyp nopt rimraf sshpk-conv sshpk-verify which
➜ app git:(main) ✗ npm i prisma
added 2 packages in 2s
3 packages are looking for funding
run `npm fund` for details
➜ app git:(main) ✗ ls node_modules/.bin
detect-libc needle node-pre-gyp prisma rc semver sshpk-sign uuid
mkdirp node-gyp nopt prisma2 rimraf sshpk-conv sshpk-verify which
➜ app git:(main) ✗ which prisma
prisma not found
但尽管如此,在程序中依旧是不可以直接用prisma
的 cli 命令的,因为这个依赖于全局prisma
。
可以考虑一下指定脚本。
不过脚本里用相对路径需要格外小心,.|..
的拼接,是相对于脚本执行者的路径而言的,而不是脚本文件!
const absDir = require('path').resolve('../../node_modules/.bin/prisma');
比如这句话,在我使用open ./MacOS/HJXH-DATA-ANALYSIS
打开时,因为脚本执行者是~
,即/Users/mark
,所以它其实以这个路径为基准,最终转换成了/node_modules/.bin/prisma
,这显然是不对的。
所以在拼接时要连接上文件路径,像这样就对了。
const absDir = require('path').resolve(
__dirname,
'../../node_modules/.bin/prisma',
);
console.log(
`database: "prisma db push", curDir: ${__dirname}, targetPrisma: ${absDir}`,
);
不过接下来又报文件找不到了,不过这是小事,马上解决。
这里总结一下:
- 在二进制程序里,用全局命令是不合适的,比较好的是用自己包里的命令,但这不像开发环境那样仿佛天生就有,具体要先
npm i prisma
安装生成.bin/prisma
可执行文件,然后再用绝对路径去引用它,当然,我们既然可以执行命令行,当然也可以导出一个变量。 - PASS: 将 prisma 命令导出,这样,在其他地方都可以用 prisma 了
指定 prisma 文件
注意看上面的报错,其实它已经提供了我们方法了,就是将schema.prisma
放到指定位置,即prisma/
下。
不过这里也能看出,他们的代码规范不统一,比如prisma.schema
在主程序中,被搜索的路径可不止这点,而且貌似还是以schema.
优先的,而不是prisma/schema.prisma
。
方法 1. 在package.json
中加入prisma.schema
字段方法 2. 移动目前的文件(从schema.prisma
到prisma/schema.prisma
)
这里就用方法 1 了:
// app/package.json
6 "prisma": {
7 "schema": "dist/main/schema.prisma"
8 },
然而还是报错了。
我验证了一下路径,是没问题的,于是顺着链接去官网看了看,突然就意识到了,这可能又是相对路径的问题。
为验证我的想法, copy 一份为prisma/schema.prisma
,预期应该还是不行,因为都是相对路径,根据官网的搜索路径./prisma/schema.prisma
就变成了~/prisma/schema.prisma
,./schema.prisma
就变成了~/schema.prisma
。
不出所料,果然!即便我路径里已经包含了两种prisma
,它都找不到,就是因为相对路径拼接错误。
➜ main git:(main) ✗ tree .
.
├── main.js
├── main.js.LICENSE.txt
├── preload.js
├── prisma
│ └── schema.prisma
└── schema.prisma
所以,我们应该自己指定路径了,既然这样,那么在package.json
中指定路径也没有意义了,因为绝对路径肯定要基于文件路径拼接的,所以只能在程序中动态拼接。
3 const _p = require("path");
4
5 const schemaExePath = _p.resolve(
6 __dirname, "../../node_modules/.bin/prisma");
7
8 const schemaFilePath = _p.resolve(__dirname, "schema.prisma");
9
10 console.log(`database: "prisma db push", curDir: ${__dirname}, targetPrismaExeP ath: ${schemaExePath}, targetPrismaFilePath: ${schemaFilePath}`);
11
12 i.default.exec(`${schemaExePath} db push --schema ${schemaFilePath}`,
prisma/client
OK!用路径拼接转绝对路径的方式,解决了上面的问题【TODO:给 prisma 提一个 pr,考虑一波基于文件的路径拼接,而非 process,要烦死了】,新的问题就是装一下prisma/client
,
这个比较简单。不过也有两种办法:
- 直接命令行安装:
npm i @prisma/client
- FAILED:尝试把外部包导入看看是不是就不需要装了
这里用第一种办法:
成了!
最后一个问题,query-engine
(2022-01-06 02:07:34 update: obviously not the last)
It seems to place the engine dirctly under th project, would b a nice solution.
A full logo about no query-engine:
01:51:49.127 › database: try to insert one iterm into `user`
01:51:49.133 › Error: Query engine library for current platform "darwin" could not be found.
You incorrectly pinned it to darwin
This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in "/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main/libquery_engine-darwin.dylib.node")
Searched Locations:
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/.prisma/client
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/node_modules/@prisma/client
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main
/tmp/prisma-engines
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main
To solve this problem, add the platform "darwin" to the "binaryTargets" attribute in the "generator" block in the "schema.prisma" file:
generator client {
provider = "prisma-client-js"
binaryTargets = ["native"]
}
Then run "prisma generate" for your changes to take effect.
Read more about deploying Prisma Client: https://pris.ly/d/client-generator
at cb (/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main/main.js:2:1130669)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
query-engine in Windows, another not positive problem
PASS: try to use keyword of native
极小自动化程序搭建
- prisma init and generate --> in order to generate target table
- change
findSync
function to fasten speed --> in order to avoid search from pc root, which would cause even minutes wait - move
schema.prisma
intorelease/app/
--> easy and default for programme search, which is consistent when in local development sinceschema.prisma
is besidesindex.js
undernode_modules/.prisma/client/
- use path-merge to generate ABSOLUTE path, both for
.bin/prisma
andschema.prisma
--> in case for the default merge behavior of prisma is wrong when the base path is offered by programme rather then by file. - install
prisma
/@prisma/client
in binary or try to pre-install
☑️ prisma
in binary
When I didn't install prisma
globally, neither had I placed prisma
into binary, then the error prisma command not found
occurred:
When I installed prisma
globally, the error then came from prisma file not found
.
So:
| install prisma globally | ✅ | ❌ |
But when I added prisma
file in the package.json
:
// package.json
"extraResources": [
"./assets/**",
"prisma/schema.prisma"
],
And it did go to the target application at: build/mac/皇家小虎快递分析系统.app/Contents/Resources/prisma/schema.prisma
The file cannot be found yet in fact.
I thought of adding into files
so that it can go to the app.asar
, but in vain:
Also failed for (in case it uses the relative path to release/app
):
"../../prisma/schema.prisma",
最大的问题,不同启动方式,程序运行结果不一样,prisma 在可执行程序中找不到
后来把程序移到/Application/
里后就变快了,甚至运行其他程序都变快了……因为它们实际用的是/Application/
里的那个schema dir
。
prisma 目前的路径设计与机制,真地适合服务端程序,因为路径比较固定,客户端是真地要疯,程序启动的位置千奇百怪,变动也很频繁,导致 prisma 那套依赖文件自动生成的机制变得复杂且难以管理。
综上,electron 不适合与 prisma 搭配,electron 这种随处运行的程序应该选择机制更灵活的一些库,比如原生的node-sqlite3
之类。
分析到此为止,和 prisma 说拜拜了!