文章目录
在Windows环境中IDEA下编译运行Elasticsearch 7.14.1
环境准备
- 在Windows环境中IDEA下编译运行Elasticsearch 7.14.1
- 环境准备
- 操作步骤
- 获取源码
- 方式一：直接通过`git bash`执行`git clone`命令
- 方式二：通过IDEA界面化操作
- 编译源码
- 本地运行
- 遇到的问题
- 1. 权限问题
- 2. 版本问题
- 3. geoip报错信息
- Cluster settings
-
JDK 16
-
Gradle 7.1
为了更快的下载，可以使用阿里的仓库服务，在解压的文件夹中的init.d文件夹下新建文件init.gradle
allprojects{
repositories {
maven{ url 'https://maven.aliyun.com/repository/public'}
maven{ url 'https://maven.aliyun.com/repository/google'}
maven{ url 'https://maven.aliyun.com/repository/gradle-plugin'}
maven{ url 'https://maven.aliyun.com/repository/jcenter'}
def REPOSITORY_URL = 'https://maven.aliyun.com/repository/public/'
all {
ArtifactRepository repo ->
if (repo instanceof MavenArtifactRepository) {
def url = repo.url.toString()
if (url.startsWith('https://artifacts.elastic.co/maven')
||url.startsWith('https://repo1.maven.org/maven2')
|| url.startsWith('https://repo.maven.org/maven2')
|| url.startsWith('https://jcenter.bintray.com/')
||url.startsWith('https://repo.maven.apache.org/maven2')
) {
project.logger.lifecycle "Repository ${repo.url} replaced by $REPOSITORY_URL."
remove repo
}
}
}
maven {
url REPOSITORY_URL
}
}
}
-
Git 2.23
-
IDEA 2021.2
-
Elasticsearch Windows版的发布包
如何确认JDK以及Gradle的版本
操作步骤 获取源码 方式一：直接通过git bash执行git clone命令
打开这个命名行工具，git clone https://github.com/elastic/elasticsearch.git，切换到7.14分支上。
通过右下角工具获取远程分支到本地
github仓库比较卡，可以使用gitee与github做映射，先同步至gitee上，在从gitee上进行clone。
编译源码用IDEA打开项目工程。
配置Gradle 
自动编译 
VM配置
-Des.path.home=E:\tmp\es\elasticsearch-7.14.1
-Des.path.conf=E:\tmp\es\elasticsearch-7.14.1\config
-Dlog4j2.disable.jmx=true
-Xmx4g
-Xms4g
[2021-09-07T17:49:40,721][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Unknown codebases [codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm] in policy file [file:/E:/code/source-code/elasticsearch7/server/out/production/resources/org/elasticsearch/bootstrap/security.policy]
Available codebases: [codebase.jna-5.7.0-1.jar, codebase.lucene-grouping, codebase.t-digest-3.2.jar, codebase.joda-time, codebase.lucene-analyzers-common-8.9.0.jar, codebase.lucene-backward-codecs, codebase.joda-time-2.10.10.jar, codebase.lucene-highlighter, codebase.log4j-api-2.11.1.jar, codebase.lucene-memory, codebase.jopt-simple, codebase.jts-core, codebase.lucene-suggest-8.9.0.jar, codebase.hppc, codebase.jna, codebase.hppc-0.8.1.jar, codebase.spatial4j-0.7.jar, codebase.lucene-join, codebase.log4j-api, codebase.HdrHistogram, codebase.jackson-dataformat-cbor, codebase.lucene-core, codebase.lucene-spatial3d-8.9.0.jar, codebase.spatial4j, codebase.log4j-core-2.11.1.jar, codebase.snakeyaml-1.26.jar, codebase.snakeyaml, codebase.jackson-dataformat-yaml-2.10.4.jar, codebase.lucene-backward-codecs-8.9.0.jar, codebase.lucene-spatial-extras-8.9.0.jar, codebase.HdrHistogram-2.1.9.jar, codebase.lucene-misc-8.9.0.jar, codebase.t-digest, codebase.lucene-highlighter-8.9.0.jar, codebase.lucene-join-8.9.0.jar, codebase.lucene-analyzers-common, codebase.jackson-dataformat-cbor-2.10.4.jar, codebase.lucene-queryparser-8.9.0.jar, codebase.lz4-java-1.8.0.jar, codebase.lucene-spatial-extras, codebase.lucene-memory-8.9.0.jar, codebase.lucene-queries-8.9.0.jar, codebase.lucene-queryparser, codebase.jts-core-1.15.0.jar, codebase.jackson-core-2.10.4.jar, codebase.lz4-java, codebase.jackson-dataformat-yaml, codebase.jackson-core, codebase.log4j-core, codebase.lucene-queries, codebase.lucene-grouping-8.9.0.jar, codebase.lucene-sandbox, codebase.lucene-suggest, codebase.lucene-sandbox-8.9.0.jar, codebase.jackson-dataformat-smile, codebase.jopt-simple-5.0.2.jar, codebase.lucene-misc, codebase.lucene-spatial3d, codebase.lucene-core-8.9.0.jar, codebase.jackson-dataformat-smile-2.10.4.jar]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[classes/:?]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[classes/:?]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[classes/:?]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[classes/:?]
at org.elasticsearch.cli.Command.main(Command.java:79) ~[classes/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[classes/:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[classes/:?]
Caused by: java.lang.IllegalArgumentException: Unknown codebases [codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm] in policy file [file:/E:/code/source-code/elasticsearch7/server/out/production/resources/org/elasticsearch/bootstrap/security.policy]
Available codebases: [codebase.jna-5.7.0-1.jar, codebase.lucene-grouping, codebase.t-digest-3.2.jar, codebase.joda-time, codebase.lucene-analyzers-common-8.9.0.jar, codebase.lucene-backward-codecs, codebase.joda-time-2.10.10.jar, codebase.lucene-highlighter, codebase.log4j-api-2.11.1.jar, codebase.lucene-memory, codebase.jopt-simple, codebase.jts-core, codebase.lucene-suggest-8.9.0.jar, codebase.hppc, codebase.jna, codebase.hppc-0.8.1.jar, codebase.spatial4j-0.7.jar, codebase.lucene-join, codebase.log4j-api, codebase.HdrHistogram, codebase.jackson-dataformat-cbor, codebase.lucene-core, codebase.lucene-spatial3d-8.9.0.jar, codebase.spatial4j, codebase.log4j-core-2.11.1.jar, codebase.snakeyaml-1.26.jar, codebase.snakeyaml, codebase.jackson-dataformat-yaml-2.10.4.jar, codebase.lucene-backward-codecs-8.9.0.jar, codebase.lucene-spatial-extras-8.9.0.jar, codebase.HdrHistogram-2.1.9.jar, codebase.lucene-misc-8.9.0.jar, codebase.t-digest, codebase.lucene-highlighter-8.9.0.jar, codebase.lucene-join-8.9.0.jar, codebase.lucene-analyzers-common, codebase.jackson-dataformat-cbor-2.10.4.jar, codebase.lucene-queryparser-8.9.0.jar, codebase.lz4-java-1.8.0.jar, codebase.lucene-spatial-extras, codebase.lucene-memory-8.9.0.jar, codebase.lucene-queries-8.9.0.jar, codebase.lucene-queryparser, codebase.jts-core-1.15.0.jar, codebase.jackson-core-2.10.4.jar, codebase.lz4-java, codebase.jackson-dataformat-yaml, codebase.jackson-core, codebase.log4j-core, codebase.lucene-queries, codebase.lucene-grouping-8.9.0.jar, codebase.lucene-sandbox, codebase.lucene-suggest, codebase.lucene-sandbox-8.9.0.jar, codebase.jackson-dataformat-smile, codebase.jopt-simple-5.0.2.jar, codebase.lucene-misc, codebase.lucene-spatial3d, codebase.lucene-core-8.9.0.jar, codebase.jackson-dataformat-smile-2.10.4.jar]
at org.elasticsearch.bootstrap.PolicyUtil.readPolicy(PolicyUtil.java:261) ~[classes/:?]
at org.elasticsearch.bootstrap.ESPolicy.(ESPolicy.java:44) ~[classes/:?]
at org.elasticsearch.bootstrap.Security.configure(Security.java:106) ~[classes/:?]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[classes/:?]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[classes/:?]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[classes/:?]
... 6 more
从错误信息看出在server模块下的security.policy文件中存在了对codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm的使用但是却有加载这几个jar
修改建议：
将关于三个jar的授权信息拷贝至公共授权方法体中，之后注释这三个jar的授权方法体
2. 版本问题[2021-09-07T17:48:05,568][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [repository-url] was built for Elasticsearch version 7.14.1 but version 7.14.2 is running
修改建议：修改文件org.elasticsearch.Version中的对象CURRENT的值为V_7_14_1。
[2021-09-07T17:18:24,147][ERROR][o.e.i.g.GeoIpDownloader ] [] exception during geoip databases update
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:608) ~[?:?]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:287) ~[?:?]
at sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]
at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:265) ~[?:?]
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372) ~[?:?]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:203) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1071) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1069) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:795) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1068) ~[?:?]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:189) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1592) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1512) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1510) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:795) ~[?:?]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) ~[?:?]
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[?:?]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334) ~[?:?]
at org.elasticsearch.ingest.geoip.HttpClient.lambda$get$0(HttpClient.java:55) ~[ingest-geoip-7.14.1.jar:7.14.1]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at org.elasticsearch.ingest.geoip.HttpClient.doPrivileged(HttpClient.java:97) ~[ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.HttpClient.get(HttpClient.java:49) ~[ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.HttpClient.getBytes(HttpClient.java:40) ~[ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.GeoIpDownloader.fetchDatabasesOverview(GeoIpDownloader.java:115) ~[ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:103) ~[ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:235) [ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:94) [ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:43) [ingest-geoip-7.14.1.jar:7.14.1]
at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:40) [classes/:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [classes/:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
此版本将GeoIp功能默认开启了采集。在默认的启动下是会去官网的默认地址下获取最新的Ip的GEO信息。 官方说明 geoip-processor
Cluster settings-
ingest.geoip.downloader.enabled(Dynamic, Boolean) If
true, Elasticsearch automatically downloads and manages updates for GeoIP2 databases from theingest.geoip.downloader.endpoint. Iffalse, Elasticsearch does not download updates and deletes all downloaded databases. Defaults totrue. -
ingest.geoip.downloader.endpoint(Static, string) Endpoint URL used to download updates for GeoIP2 databases. Defaults to
https://geoip.elastic.co/v1/database. Elasticsearch stores downloaded database files in each node’s temporary directory at$ES_TMPDIR/geoip-databases/. -
ingest.geoip.downloader.poll.interval(Dynamic, time value) How often Elasticsearch checks for GeoIP2 database updates at the
ingest.geoip.downloader.endpoint. Must be greater than1d(one day). Defaults to3d(three days).
修改建议：
修改es发布包解压后的文件夹elasticsearch-7.14.1\config中的elasticsearch.yml文件，新增配置
ingest.geoip.downloader.enabled: false
启动，check集群健康状态 
至此启动完成，可以进行源码的运行阅读分析了。
