跳到主要内容
版本:0.17.0+

electron prisma sqlite 2

[toc]

PASS: test putting prisma | @prisma/client under release package.json

✅ cannot start when integrated with sqlite in packed version

suppose:

APP=release/app

step 1. add dependency in APP/package.json

// release/app/package.json
"dependencies": {
"sqlite3": "^5.0.2"
},

step 2. rebuild project

:::warn maybe we should node i -g node-gyp first :::

[npm run rebuild] electron-rebuild --parallel --types prod,dev,optional --module-dir $APP

step 3. build and pack again

npm run build
[npm run app:dir] electron-builder --dir

ref:

step 4. PLUS: if we don't pack sqlite3 into $APP, then we won't pass the webpack build process only if we add this

// .erb/configs/webpack.config.base.ts
externals: [
...Object.keys(externals || {}),

// for prisma build
// https://github.com/prisma/prisma/issues/6564#issuecomment-899013495
{
_http_common: '_http_common',
},

// for sqlite3 build
// https://stackoverflow.com/a/51256562/9422455
{ sqlite3: 'commonjs sqlite3' },
],

✅ can normally open if no database connection in source code

// src/main/db.ts
interface FakePrisma {
erp: {
create: any;
upsert: any;
findMany: any;
};
}

export const prisma: FakePrisma = {
erp: {
create: () => {
console.log('fake-database: creating one');
},
findMany: () => {
console.log('fake-database: finding many');
},
upsert: () => {
console.log('fake-database: upserting one');
},
},
};

using pure sqlite3 and the programme goes well under prod, then failed with a popup suggesting me to integrate sqlite3

picture 11

integrate prisma like sqlite

When add the "@prisma/client": "^3.7.0" into APP/package.json: picture 12

So we then try to add .prisma into it.

  "dependencies": {
"@prisma/client": "^3.7.0",
"sqlite3": "^5.0.2"
},
"devDependencies": {
"prisma": "^3.7.0",
"@prisma/client": "^3.7.0"
}

picture 13picture 14picture 15

After I moved the libxxxxnode into main, then things are ok, except still too slow!

picture 16

I try to debug as Other packages make Prisma load very slow · Issue #8484 · prisma/prisma did:

picture 17

I found this app used my local packages, which maybe the issue behind the load problem.

So I uninstalled the global prisma:

npm uninstall -g prisma

Then I run again.

picture 18

However, it starts to use the packages in my dev project.

Finally I do a copy before I run for test:

# $erb
# copy
cp prisma/schema.prisma $app/Contents/Resources/app/dist/main/
cp node_modules/.prisma/client/libquery_engine-darwin.dylib.node $app/Contents/Resources/app/dist/main

# run
open release/build/mac/皇家小虎快递分析系统.app/Contents/MacOS/皇家小虎快递分析系统

✨ maybe it's because prisma is searching file location, so that the system runs slow! (since finally it didn't search any schema.prisma, which enlightened me!)

I am quite sure now the problem is about package import with prisma

I added one log just at the entrance of the whole programme.

picture 20

And I am lucky to see it is printed after programme runs in binary, and then wait, which indicates the start of programme is ok, except the packages.

picture 19

Yeah, thanks Mac's console, I saw the dawn ✨✨:sparkles

Stupid!

I found nothing helpful!

(Since the programme didn't cause any system exception, except block)

Finally found the problem, it's about the block of searching path

Just add these two lines for debug the findSync sentence. picture 24

Then you would know where does it go wrong. picture 23

I located to the source code. picture 25

It seems the path generated incorrectly.

Since this directory is generated via cli, I re-generate it, which looks more normal then.

picture 26

And then we pack again.

npm run build
npm run app:dir

Look into what's new in the APP/../main.js picture 27

OK, it's consistent with the source code after we prisma generate.

However, there's no node_modules neither .prisma under main.js directory now:

➜  main git:(main) ✗ ls
main.js main.js.LICENSE.txt preload.js

It's because we didn't pack it into app.asar in electron.

So how to achieve this goal?

  1. we cannot use externalResources choice since it only pack files into Resources/ not app/main/
  2. if we use files, then the files would list in app.asar under app directory, this is what we are looking for.
  3. if we add the dependencies under app/package.json, then there would be node_modules installed under app. It is also suitable if there would generate a .prisma/client, we can give it a shot only if we add @prisma/client into app/package.json.
// release/app/package.json
"dependencies": {
"sqlite3": "^5.0.2",
"@prisma/client": "^3.7.0"
}

However, I tried but to find there is NO .prisma folder in the target place.

And I tried locally to find that if I do not use prisma generate then I only got an blank template index.js, which is certainly not I want.

picture 28

So, the answer is clear now: we can't put @prisma/client into app/package.json, since it won't generate the right sql model unless we use prisma generate. And even the .prisma generated locally, it won't be packed into out target application's app.asar folder.

Therefore, the only way best suitable for us is to manually copy the local .prisma folder to app/dist/main/ so that can be searched by prisma in the binary application.

We then use prisma generate to synchronize the node_modules/.prisma/client/index.js with out prisma/schema.prisma file (which would copied into node_modules/.prisma/client/schema.prisma), along with the generated query engine for MacOS: node_modules/.prisma/client/libquery_engine-darwin.dylib.node.[TODO: what should windows user do; maybe copy all the files into app is not ideal for cross platform use. Instead, we should consider about only copy the schema.prisma file, and let all the other work generated by platforms]

picture 29

I finally understand why programme start by double click would be so slow

I added log before findSync again, and run by double click of the main programme.

picture 32

And it should tell me that the process.cwd() is / that means from my computer root!

picture 30

As a contrast, I ran it according the electron helper which is within the main programme.

And the process.cwd() is ~, which is much much faster! picture 33

what's the findSync

picture 34

So, according the source code, the findSync starts from the root, then to search target pattern.

picture 35

Thus, from the commands before, we can see, it would search target pattern node_modules/.prisma/client or .prisma/client from process.cwd() (i.e. / when run main programme and ~ when run in shell)

What's the solution?

  1. directly change the logic into _ = __dirname, and put .prisma/client under the __dirname, i.e. app/main

  2. perhaps there's way (i.e. package or not) to not go this logic, which is so-called best practice? But sadly, I haven't found one after many times of hard search in github of prisma. I am looking forward the developers of prisma can give one.

    1. For example, maybe there is a way allows me to fill some parameters so that to disable the hard search or change the root from?
picture 37

ref:

Current Solution: avoid `findSync'

avoid any call of findSync

// node_modules/.prisma/client/index.js

// to save the hard search time(maybe from '/'), let us directly point out the target prisma client place
console.log({process_cwd: process.cwd(), __dirname});
const dirname = path.join(__dirname, '.prisma/client');
console.log({dirname});

// const { findSync } = require('@prisma/client/runtime')
//
// const dirname = findSync(process.cwd(), [
// "node_modules/.prisma/client",
// ".prisma/client",
// ], ['d'], ['d'], 1)[0] || __dirname
// main files structure of an packed electron on Mac
// with `.prisma/client` besides `main.js`
// generate via ` find . ! -path '*node_modules/*' ! -path '*Frameworks/*' ! -path '*assets*' ! -path '*lproj' ! -path '*svg' | tree --fromfile --noreport -aC`

. // Main Entrance
└── Contents
├── Frameworks
├── Info.plist
├── MacOS
│   └── HJXH-DATA-ANALYSIS // Script Entrance
├── PkgInfo
└── Resources
├── app
│   ├── dist
│   │   ├── main
│   │   │   ├── .prisma
│   │   │   │   └── client
│   │   │   │   ├── index-browser.js
│   │   │   │   ├── index.js
│   │   │   │   ├── libquery_engine-darwin.dylib.node
│   │   │   │   ├── package.json
│   │   │   │   └── schema.prisma
│   │   │   ├── main.js // Main Runner
│   │   │   ├── main.js.LICENSE.txt
│   │   │   └── preload.js
│   │   └── renderer
│   │   ├── index.html
│   │   ├── renderer.js
│   │   └── renderer.js.LICENSE.txt
│   ├── node_modules
│   └── package.json
└── icon.icns

make binary can use prisma db push

The question is to intercept prisma into package use, am I right?(maybe not...)

Anyway, we'd better have a try at the first: add prisma as a dependency of app.

picture 36

Now, we have prisma in node_modules.

However, I doesn't be positive about running with prisma since it's only a dependency not a command.

There are at least two kinds of commands we can use, one for global, one for those under node_modules/.bin.

The sad truth is that only when we run npm i prisma in packed app, we can have a prisma under node_modules/.bin, which seems unacceptable. [TODO: so why they cannot generate a prisma under .bin when I have assigned prisma in package.json of released app]

➜  app git:(main) ✗ npm i prisma

added 2 packages, and changed 1 package in 2s

3 packages are looking for funding
run `npm fund` for details
➜ app git:(main) ✗ ls node_modules/.bin
detect-libc node-gyp prisma rimraf sshpk-sign which
mkdirp node-pre-gyp prisma2 semver sshpk-verify
needle nopt rc sshpk-conv uuid

So let's have a try step by step.

First, only add prisma in the package.json of release.

若在release/package.json里指定了@prisma/client,则打包里没有.prisma/client/index.js

picture 38

若未指定(只指定一个sqlite3),则有

picture 39

若直接指定两个(2022-01-06 02:15:12),则像一个一样还是报错了,但是我联想到了rebuild

picture 12

So when I used rebuild (in fact, it's npm run post-install instead of npm run rebuild), everything is okay now! (although with a lot of errors...)

picture 13

rebuild 确实是一大解药,rebuild完之后还要把prisma移到对应的node_modules目录旁,运行一遍prisma generate,目前我一套流程走下来,将prisma一步步同步,效果还行;只不过最后一步还是有问题,感觉双击程序启动找不到路径的问题还是存在。既然这样,步步同步的方式似乎也没啥性价比了

再测试一下步步同步的方式是否可以装 windows,如果也不行,我就直接放弃 prisma 了

确实不行,只有darwin,没有看到给win用的 query-engine.

在 binary 里需要用npm i prisma安装脚本

➜  app git:(main) ✗ ls node_modules/.bin
detect-libc needle node-pre-gyp rc semver sshpk-sign uuid
mkdirp node-gyp nopt rimraf sshpk-conv sshpk-verify which
➜ app git:(main) ✗ npm i prisma

added 2 packages in 2s

3 packages are looking for funding
run `npm fund` for details
➜ app git:(main) ✗ ls node_modules/.bin
detect-libc needle node-pre-gyp prisma rc semver sshpk-sign uuid
mkdirp node-gyp nopt prisma2 rimraf sshpk-conv sshpk-verify which
➜ app git:(main) ✗ which prisma
prisma not found

但尽管如此,在程序中依旧是不可以直接用prisma的 cli 命令的,因为这个依赖于全局prismapicture 40

可以考虑一下指定脚本。

picture 41

不过脚本里用相对路径需要格外小心,.|..的拼接,是相对于脚本执行者的路径而言的,而不是脚本文件!

const absDir = require('path').resolve('../../node_modules/.bin/prisma');

比如这句话,在我使用open ./MacOS/HJXH-DATA-ANALYSIS打开时,因为脚本执行者是~,即/Users/mark,所以它其实以这个路径为基准,最终转换成了/node_modules/.bin/prisma,这显然是不对的。

picture 42

所以在拼接时要连接上文件路径,像这样就对了。

const absDir = require('path').resolve(
__dirname,
'../../node_modules/.bin/prisma',
);
console.log(
`database: "prisma db push", curDir: ${__dirname}, targetPrisma: ${absDir}`,
);
picture 43

不过接下来又报文件找不到了,不过这是小事,马上解决。

这里总结一下:

  1. 在二进制程序里,用全局命令是不合适的,比较好的是用自己包里的命令,但这不像开发环境那样仿佛天生就有,具体要先npm i prisma安装生成.bin/prisma可执行文件,然后再用绝对路径去引用它,当然,我们既然可以执行命令行,当然也可以导出一个变量。
  2. PASS: 将 prisma 命令导出,这样,在其他地方都可以用 prisma 了

指定 prisma 文件

注意看上面的报错,其实它已经提供了我们方法了,就是将schema.prisma放到指定位置,即prisma/下。

不过这里也能看出,他们的代码规范不统一,比如prisma.schema在主程序中,被搜索的路径可不止这点,而且貌似还是以schema.优先的,而不是prisma/schema.prisma

方法 1. 在package.json中加入prisma.schema字段方法 2. 移动目前的文件(从schema.prismaprisma/schema.prisma

这里就用方法 1 了:

// app/package.json
6 "prisma": {
7 "schema": "dist/main/schema.prisma"
8 },

然而还是报错了。

picture 45

我验证了一下路径,是没问题的,于是顺着链接去官网看了看,突然就意识到了,这可能又是相对路径的问题。

picture 44

为验证我的想法, copy 一份为prisma/schema.prisma,预期应该还是不行,因为都是相对路径,根据官网的搜索路径./prisma/schema.prisma就变成了~/prisma/schema.prisma./schema.prisma就变成了~/schema.prisma

不出所料,果然!即便我路径里已经包含了两种prisma,它都找不到,就是因为相对路径拼接错误。

➜  main git:(main) ✗ tree .
.
├── main.js
├── main.js.LICENSE.txt
├── preload.js
├── prisma
│   └── schema.prisma
└── schema.prisma

所以,我们应该自己指定路径了,既然这样,那么在package.json中指定路径也没有意义了,因为绝对路径肯定要基于文件路径拼接的,所以只能在程序中动态拼接。

  3 const _p = require("path");
4
5 const schemaExePath = _p.resolve(
6 __dirname, "../../node_modules/.bin/prisma");
7
8 const schemaFilePath = _p.resolve(__dirname, "schema.prisma");
9
10 console.log(`database: "prisma db push", curDir: ${__dirname}, targetPrismaExeP ath: ${schemaExePath}, targetPrismaFilePath: ${schemaFilePath}`);
11
12 i.default.exec(`${schemaExePath} db push --schema ${schemaFilePath}`,

prisma/client

OK!用路径拼接转绝对路径的方式,解决了上面的问题【TODO:给 prisma 提一个 pr,考虑一波基于文件的路径拼接,而非 process,要烦死了】,新的问题就是装一下prisma/client

picture 46

这个比较简单。不过也有两种办法:

  1. 直接命令行安装:npm i @prisma/client
  2. FAILED:尝试把外部包导入看看是不是就不需要装了

这里用第一种办法:

picture 47

成了!

最后一个问题,query-engine (2022-01-06 02:07:34 update: obviously not the last)

It seems to place the engine dirctly under th project, would b a nice solution.

picture 48

A full logo about no query-engine:

01:51:49.127 › database: try to insert one iterm into `user`
01:51:49.133 › Error: Query engine library for current platform "darwin" could not be found.
You incorrectly pinned it to darwin

This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in "/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main/libquery_engine-darwin.dylib.node")

Searched Locations:

/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/.prisma/client
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/node_modules/@prisma/client
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main
/tmp/prisma-engines
/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main


To solve this problem, add the platform "darwin" to the "binaryTargets" attribute in the "generator" block in the "schema.prisma" file:
generator client {
provider = "prisma-client-js"
binaryTargets = ["native"]
}

Then run "prisma generate" for your changes to take effect.
Read more about deploying Prisma Client: https://pris.ly/d/client-generator
at cb (/Users/mark/Documents/mark_projects/hjxh/hjxh_express_match/erb/release/build/mac/HJXH-DATA-ANALYSIS.app/Contents/Resources/app/dist/main/main.js:2:1130669)
at processTicksAndRejections (node:internal/process/task_queues:96:5)

query-engine in Windows, another not positive problem

picture 11

PASS: try to use keyword of native

极小自动化程序搭建

  1. prisma init and generate --> in order to generate target table
  2. change findSync function to fasten speed --> in order to avoid search from pc root, which would cause even minutes wait
  3. move schema.prisma into release/app/ --> easy and default for programme search, which is consistent when in local development since schema.prisma is besides index.js under node_modules/.prisma/client/
  4. use path-merge to generate ABSOLUTE path, both for .bin/prisma and schema.prisma --> in case for the default merge behavior of prisma is wrong when the base path is offered by programme rather then by file.
  5. install prisma/@prisma/client in binary or try to pre-install

☑️ prisma in binary

picture 5

When I didn't install prisma globally, neither had I placed prisma into binary, then the error prisma command not found occurred:

picture 7

When I installed prisma globally, the error then came from prisma file not found.

picture 6

So:

| install prisma globally | ✅ | ❌ |

But when I added prisma file in the package.json:

// package.json
"extraResources": [
"./assets/**",
"prisma/schema.prisma"
],

And it did go to the target application at: build/mac/皇家小虎快递分析系统.app/Contents/Resources/prisma/schema.prisma

The file cannot be found yet in fact.

picture 8

I thought of adding into files so that it can go to the app.asar, but in vain:

picture 9

Also failed for (in case it uses the relative path to release/app):

     "../../prisma/schema.prisma",

最大的问题,不同启动方式,程序运行结果不一样,prisma 在可执行程序中找不到

picture 14

后来把程序移到/Application/里后就变快了,甚至运行其他程序都变快了……因为它们实际用的是/Application/里的那个schema dir

prisma 目前的路径设计与机制,真地适合服务端程序,因为路径比较固定,客户端是真地要疯,程序启动的位置千奇百怪,变动也很频繁,导致 prisma 那套依赖文件自动生成的机制变得复杂且难以管理。

综上,electron 不适合与 prisma 搭配,electron 这种随处运行的程序应该选择机制更灵活的一些库,比如原生的node-sqlite3之类。

分析到此为止,和 prisma 说拜拜了!