Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 6.0: added multi thread supports #5320

Merged
merged 105 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
3a17c34
Optimize code
matyhtf Mar 15, 2024
9bc606b
thread local
matyhtf Mar 15, 2024
908644c
swoole thread
matyhtf Mar 19, 2024
7c3cb35
optimize argv
matyhtf Mar 19, 2024
c190fc8
optimize code, add more methods for thread
matyhtf Mar 20, 2024
4ad6b36
Swoole\Thread\Map
matyhtf Mar 20, 2024
ef5fb06
Swoole\Thread\ArrayList
matyhtf Mar 20, 2024
a721e65
Optimize
matyhtf Mar 20, 2024
cc47833
zend_array
matyhtf Mar 20, 2024
604b35e
Optimize
matyhtf Mar 21, 2024
f1dbc6a
Optimize
matyhtf Mar 21, 2024
55d2c57
Optimize
matyhtf Mar 21, 2024
44f1c4c
fix
matyhtf Mar 21, 2024
c366385
remove co redis/mysql/pgsql client
matyhtf Mar 25, 2024
6969a36
Added Thread\Queue
matyhtf Mar 25, 2024
fe199d7
Fix Thread\Queue
matyhtf Mar 26, 2024
112d692
fix mem leak
matyhtf Mar 26, 2024
449adf9
optimize, reduce memory copy
matyhtf Mar 26, 2024
88797a5
Revert "optimize, reduce memory copy"
matyhtf Mar 26, 2024
b6a2c22
optimize code, remove 8.0 supports
matyhtf Mar 26, 2024
684e419
refactor atomic/lock, support thread
matyhtf Mar 26, 2024
afae718
clang-format
matyhtf Mar 27, 2024
143f71e
fix tests
matyhtf Mar 27, 2024
e3a29e6
[6.0] Server for thread mode (#5282)
matyhtf Mar 28, 2024
bfe2ff2
Merge branch 'master' into v6.0
matyhtf Mar 28, 2024
fe53be3
Update version
matyhtf Mar 28, 2024
5b00750
Optimize header
matyhtf Mar 28, 2024
54071d0
Fix
matyhtf Mar 28, 2024
b6962ab
refactor async-threads, support ZTS
matyhtf Mar 29, 2024
bc9b8f8
Optimize
matyhtf Mar 29, 2024
80f2d92
Optimize
matyhtf Apr 1, 2024
b2bf40a
Optimize
matyhtf Apr 1, 2024
dc46529
Optimize
matyhtf Apr 1, 2024
98570da
Optimize 5
matyhtf Apr 1, 2024
a4ed205
Optimize 6
matyhtf Apr 1, 2024
3b50c71
Optimize 7
matyhtf Apr 1, 2024
4f992f7
Optimize 8
matyhtf Apr 1, 2024
0631581
Optimize 10
matyhtf Apr 2, 2024
8378690
Optimize 11, add Server::get_worker_id()
matyhtf Apr 2, 2024
01275cf
Optimize 12
matyhtf Apr 3, 2024
df62cc3
Optimize 13
matyhtf Apr 3, 2024
053a7d4
Optimize 14
matyhtf Apr 7, 2024
7f63f02
rename
matyhtf Apr 7, 2024
aeeb79f
Refactor
matyhtf Apr 7, 2024
b2fe971
Refactor 2
matyhtf Apr 8, 2024
993bcdb
revert , format
matyhtf Apr 8, 2024
e824c86
onPipeMessage/onTask/onFinish, fix message bus
matyhtf Apr 8, 2024
5d0df0e
Update README.md
matyhtf Apr 9, 2024
319bba5
Added pty support for proc_open function, fix #5275 (#5290)
matyhtf Apr 9, 2024
ed3af5b
Update boost asm (#5291)
NathanFreeman Apr 10, 2024
34d9616
Merge branch 'v6.0' of github.com:swoole/swoole-src into v6.0
matyhtf Apr 10, 2024
cc6cd14
fix tests
matyhtf Apr 10, 2024
907269c
fix tests
matyhtf Apr 10, 2024
caa4aeb
fix tests
matyhtf Apr 11, 2024
ab2eea8
remove hiredis
matyhtf Apr 11, 2024
9819341
fix tests [3]
matyhtf Apr 11, 2024
f57d033
fix tests [4]
matyhtf Apr 12, 2024
f4cb996
fix tests [5], revert SwooleWG
matyhtf Apr 12, 2024
acbd5be
fix tests [6]
matyhtf Apr 12, 2024
b78ef3e
fix tests [7]
matyhtf Apr 12, 2024
1d5046f
fix tests [8]
matyhtf Apr 12, 2024
d3eae1d
optimize tests
matyhtf Apr 15, 2024
a538cbc
add thread test
matyhtf Apr 15, 2024
bdfafcc
fix ci
matyhtf Apr 15, 2024
b6b529f
[test] fix ci 2
matyhtf Apr 15, 2024
1c017a4
[test] fix ci 3
matyhtf Apr 15, 2024
35bd09f
[test] fix ci 4
matyhtf Apr 15, 2024
d0fbc15
[test] fix ci 5
matyhtf Apr 15, 2024
23c5d7b
[test] fix ci 6
matyhtf Apr 15, 2024
1f93781
[test] fix ci 7
matyhtf Apr 15, 2024
a20ccda
[test] fix ci 8
matyhtf Apr 15, 2024
70efa87
[test] fix ci 9
matyhtf Apr 15, 2024
c5e55f6
[test] fix ci 10
matyhtf Apr 15, 2024
1fbbd49
[test] fix ci 11
matyhtf Apr 15, 2024
b20cb3c
[test] remove swoole_timer_set, async settings can only be set in the…
matyhtf Apr 15, 2024
1594cf5
optimize create socket (#5293)
NathanFreeman Apr 16, 2024
20eec88
[test] fix tests
matyhtf Apr 16, 2024
d66eabf
[test] fix tests [3]
matyhtf Apr 16, 2024
93d6cec
[test] fix tests [4]
matyhtf Apr 17, 2024
14b0406
sync plain_wrapper (#5296)
NathanFreeman Apr 19, 2024
0f85227
[test] fix tests [5]
matyhtf Apr 19, 2024
d0e8154
No limit on the maximum buffer length of the read pipeline
matyhtf Apr 23, 2024
2589cfc
Refactor
matyhtf Apr 23, 2024
5cc516b
fix
matyhtf Apr 23, 2024
8b95d8b
optimize tests
matyhtf Apr 23, 2024
0168ebe
fix tests
matyhtf Apr 23, 2024
f6fcafb
code format
matyhtf Apr 23, 2024
69effb7
fix tests
matyhtf Apr 23, 2024
ed50730
fix tests
matyhtf Apr 23, 2024
0890024
fix tests [3]
matyhtf Apr 23, 2024
afecffc
Fix transfer_t struct missing (#5303)
NathanFreeman Apr 23, 2024
0a2671f
Optimize thread context
matyhtf Apr 23, 2024
c2b4472
Fix tests[4]
matyhtf Apr 23, 2024
cf0f1db
Refactor
matyhtf Apr 24, 2024
92afad3
Refactor
matyhtf Apr 25, 2024
44b3d0f
Refactor
matyhtf Apr 25, 2024
9fd4829
fix server shutdown
matyhtf Apr 25, 2024
04c421e
fix thread atomic
matyhtf Apr 25, 2024
bd28829
fix core tests
matyhtf Apr 25, 2024
0932447
Fix Sqlite pdo segmentfault (#5311)
NathanFreeman May 6, 2024
e30f405
fix core tests
matyhtf May 6, 2024
f857737
Support passing streams between threads
matyhtf May 6, 2024
108ee2a
Support using stream as a thread argument
matyhtf May 6, 2024
b0cafc8
optimize code
matyhtf May 6, 2024
9d59862
optimize code
matyhtf May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 2 additions & 1 deletion .github/workflows/conflict-exts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@ on: [push, pull_request]

jobs:
tests:
if: "github.repository_owner == 'swoole' && !contains(github.event.head_commit.message, '[test]')"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ['8.0', '8.1', '8.2', '8.3']
php: ['8.1', '8.2', '8.3']

name: PHP ${{ matrix.php }} - Swoole

Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/framework.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ on:
jobs:
linux:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, '[test]')"
if: 0
# if: "!contains(github.event.head_commit.message, '[test]')"
strategy:
fail-fast: false
matrix:
php-version: [ '8.0', '8.1', '8.2', '8.3' ]
php-version: [ '8.1', '8.2', '8.3' ]
framework: [ 'Laravel Octane', 'Hyperf', 'Simps', 'imi' ]
name: ${{ matrix.framework }} - PHP ${{ matrix.php-version }}
steps:
Expand Down Expand Up @@ -102,7 +103,7 @@ jobs:
strategy:
fail-fast: false
matrix:
php-version: [ '8.0', '8.1', '8.2', '8.3' ]
php-version: [ '8.1', '8.2', '8.3' ]
framework: [ 'Simps' ]
name: ${{ matrix.framework }} - PHP ${{ matrix.php-version }} - macOS
steps:
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/test-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ on: [push, pull_request]
jobs:
test-linux:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, '[zts]')"
strategy:
fail-fast: false
matrix:
php: ['8.0', '8.1', '8.2', '8.3']
php: ['8.1', '8.2', '8.3']
steps:
- uses: actions/checkout@v4
- name: Setup PHP
Expand All @@ -34,4 +35,6 @@ jobs:
- name: Run Swoole test
run: |
export SWOOLE_BRANCH=${GITHUB_REF##*/}
export SWOOLE_BUILD_DIR=$(realpath .)
export PHP_VERSION=${{ matrix.php }}
${{runner.workspace}}/swoole-src/scripts/route.sh
41 changes: 41 additions & 0 deletions .github/workflows/thread.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Thread Support Tests

on: [push, pull_request]

jobs:
test-linux:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, '[nts]')"
strategy:
fail-fast: false
matrix:
php: ['8.1-zts', '8.2-zts', '8.3-zts']
name: ${{ matrix.php }}
steps:
- uses: actions/checkout@v4
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: "${{ matrix.php }}"
coverage: none
env:
phpts: ts
- name: Show machine information
run: |
date
env
uname -a
ulimit -a
php -v
php --ini
ls -al
pwd
echo "`git log -20 --pretty --oneline`"
echo "`git log -10 --stat --pretty --oneline`"
- name: Run Swoole test
run: |
export SWOOLE_BRANCH=${GITHUB_REF##*/}
export SWOOLE_THREAD=1
export SWOOLE_BUILD_DIR=$(realpath .)
export PHP_VERSION=${{ matrix.php }}
${SWOOLE_BUILD_DIR}/scripts/route.sh
4 changes: 2 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
PROJECT(libswoole)
cmake_minimum_required(VERSION 2.8.12)

ENABLE_LANGUAGE(ASM)
set(SWOOLE_VERSION 5.1.2)
set(SWOOLE_VERSION 6.0.0-dev)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -g")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall")
cmake_minimum_required(VERSION 2.8)

file(READ ./config.h SWOOLE_CONFIG_FILE)

Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ docker run --rm phpswoole/swoole "php --ri swoole"

> For details on how to use it, see: [How to Use This Image](https://github.com/swoole/docker-swoole#how-to-use-this-image).

## Documentation
<https://wiki.swoole.com/>

### HTTP Service
```php
$http = new Swoole\Http\Server('127.0.0.1', 9501);
Expand Down Expand Up @@ -115,7 +118,6 @@ Co\run(function() {
+ __IDE Helper & API__: <https://github.com/swoole/ide-helper>
+ __Twitter__: <https://twitter.com/phpswoole>
+ __Discord__: <https://discord.swoole.dev>
+ __中文文档__: <https://wiki.swoole.com>
+ __中文社区__: <https://wiki.swoole.com/#/other/discussion>

## 💎 Awesome Swoole
Expand Down
32 changes: 20 additions & 12 deletions config.m4
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,11 @@ PHP_ARG_ENABLE([thread-context],
[AS_HELP_STRING([--enable-thread-context],
[Use thread context])], [no], [no])

PHP_ARG_ENABLE([swoole-thread],
[whether to enable swoole thread support],
[AS_HELP_STRING([--enable-swoole-thread],
[Enable swoole thread support])], [no], [no])

PHP_ARG_ENABLE([swoole-coro-time],
[whether to enable coroutine execution time ],
[AS_HELP_STRING([--enable-swoole-coro-time],
Expand Down Expand Up @@ -877,6 +882,10 @@ EOF
AC_DEFINE(SW_LOG_TRACE_OPEN, 1, [enable trace log])
fi

if test "$PHP_SWOOLE_THREAD" != "no"; then
AC_DEFINE(SW_THREAD, 1, [enable swoole thread support])
fi

if test "$PHP_SOCKETS" = "yes"; then
AC_MSG_CHECKING([for php_sockets.h])

Expand Down Expand Up @@ -1025,13 +1034,6 @@ EOF
thirdparty/swoole_http_parser.c \
thirdparty/multipart_parser.c"

swoole_source_file="$swoole_source_file \
thirdparty/hiredis/hiredis.c \
thirdparty/hiredis/alloc.c \
thirdparty/hiredis/net.c \
thirdparty/hiredis/read.c \
thirdparty/hiredis/sds.c"

if test "$PHP_NGHTTP2_DIR" = "no"; then
PHP_ADD_INCLUDE([$ext_srcdir/thirdparty])
swoole_source_file="$swoole_source_file \
Expand Down Expand Up @@ -1073,7 +1075,9 @@ EOF
thirdparty/php80/pdo_sqlite/sqlite_driver.c \
thirdparty/php80/pdo_sqlite/sqlite_statement.c \
thirdparty/php81/pdo_sqlite/sqlite_driver.c \
thirdparty/php81/pdo_sqlite/sqlite_statement.c"
thirdparty/php81/pdo_sqlite/sqlite_statement.c \
thirdparty/php83/pdo_sqlite/sqlite_driver.c \
thirdparty/php83/pdo_sqlite/sqlite_statement.c"
fi

SW_ASM_DIR="thirdparty/boost/asm/"
Expand All @@ -1090,6 +1094,7 @@ EOF
[mips64*], [SW_CPU="mips64"],
[mips*], [SW_CPU="mips32"],
[riscv64*], [SW_CPU="riscv64"],
[loongarch64*], [SW_CPU="loongarch64"],
[
SW_USE_ASM_CONTEXT="no"
]
Expand Down Expand Up @@ -1151,6 +1156,12 @@ EOF
else
SW_USE_ASM_CONTEXT="no"
fi
elif test "$SW_CPU" = "loongarch64"; then
if test "$SW_OS" = "LINUX"; then
SW_CONTEXT_ASM_FILE="loongarch64_sysv_elf_gas.S"
else
SW_USE_ASM_CONTEXT="no"
fi
else
SW_USE_ASM_CONTEXT="no"
fi
Expand All @@ -1175,7 +1186,6 @@ EOF
PHP_ADD_INCLUDE([$ext_srcdir/include])
PHP_ADD_INCLUDE([$ext_srcdir/ext-src])
PHP_ADD_INCLUDE([$ext_srcdir/thirdparty])
PHP_ADD_INCLUDE([$ext_srcdir/thirdparty/hiredis])

AC_MSG_CHECKING([swoole coverage])
if test "$PHP_SWOOLE_COVERAGE" != "no"; then
Expand All @@ -1190,8 +1200,7 @@ EOF
include/*.h \
stubs/*.h \
thirdparty/*.h \
thirdparty/nghttp2/*.h \
thirdparty/hiredis/*.h])
thirdparty/nghttp2/*.h])

PHP_REQUIRE_CXX()

Expand Down Expand Up @@ -1220,7 +1229,6 @@ EOF
PHP_ADD_BUILD_DIR($ext_builddir/src/wrapper)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/boost)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/boost/asm)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/hiredis)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/php/sockets)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/php/standard)
PHP_ADD_BUILD_DIR($ext_builddir/thirdparty/php/curl)
Expand Down
8 changes: 4 additions & 4 deletions core-tests/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 2.8.1)
cmake_minimum_required(VERSION 2.8.12)

project(core_tests)

Expand All @@ -10,14 +10,14 @@ file(GLOB_RECURSE SOURCE_FILES FOLLOW_SYMLINKS src/*.cpp deps/llhttp/src/*.c)

add_definitions(-DHAVE_CONFIG_H)

set(core_tests_includes ./include/ ../thirdparty ../thirdparty/hiredis ./deps/llhttp/include)
set(core_tests_includes ./include/ ../thirdparty ../thirdparty/hiredis ./deps/llhttp/include /usr/local/include)
set(core_tests_libraries)
set(core_tests_link_directories /usr/local/lib)

list(APPEND core_tests_libraries pthread)
list(APPEND core_tests_libraries pthread gtest gtest_main)

# find GTest
find_package(GTest)
find_package(GTest REQUIRED)
if (!${GTEST_FOUND})
message(FATAL_ERROR "Not found GTest")
endif()
Expand Down
8 changes: 7 additions & 1 deletion core-tests/src/os/async.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,16 @@ TEST(async, schedule) {
count--;
if (count == 0) {
swoole_timer_del(timer);
ASSERT_EQ(SwooleTG.async_threads->get_worker_num(), 128);
ASSERT_GT(SwooleTG.async_threads->get_worker_num(), 16);
ASSERT_GT(SwooleTG.async_threads->get_queue_size(), 100);
ASSERT_GT(SwooleTG.async_threads->get_task_num(), 100);
break;
} else if (count == N - 1) {
ASSERT_EQ(SwooleTG.async_threads->get_worker_num(), 4);
ASSERT_EQ(SwooleTG.async_threads->get_queue_size(), 1);
ASSERT_EQ(SwooleTG.async_threads->get_task_num(), 1);
} else if (count < N / 2) {
ASSERT_GT(SwooleTG.async_threads->get_worker_num(), 4);
}
}
});
Expand Down
2 changes: 1 addition & 1 deletion core-tests/src/server/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,7 @@ TEST(server, task_worker4) {
serv->gs->task_workers.dispatch(&buf, &_dst_worker_id);
sleep(1);

EventData *task_result = &(serv->task_result[SwooleG.process_id]);
EventData *task_result = &(serv->task_result[swoole_get_process_id()]);
sw_memset_zero(task_result, sizeof(*task_result));
memset(&buf.info, 0, sizeof(buf.info));
buf.info.len = strlen(packet);
Expand Down
42 changes: 42 additions & 0 deletions examples/thread/aio.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<?php

use Swoole\Thread;
use Swoole\Thread\Queue;


$args = Thread::getArguments();
$c = 4;
$running = true;

if (empty($args)) {
$threads = [];
$atomic = new Swoole\Thread\Atomic();
for ($i = 0; $i < $c; $i++) {
$threads[] = Thread::exec(__FILE__, $i, $atomic);
}
for ($i = 0; $i < $c; $i++) {
$threads[$i]->join();
}
var_dump($atomic->get());
sleep(2);

Co\run(function () use($atomic) {
$n = 1024;
while ($n--) {
$atomic->add();
$rs = \Swoole\Coroutine\System::readFile(__FILE__);
var_dump(strlen($rs));
}
});
var_dump($atomic->get());
} else {
$atomic = $args[1];
Co\run(function () use($atomic) {
$n = 1024;
while ($n--) {
$atomic->add();
$rs = \Swoole\Coroutine\System::readFile(__FILE__);
var_dump(strlen($rs));
}
});
}
14 changes: 14 additions & 0 deletions examples/thread/argv.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?php

use Swoole\Thread;

$args = Thread::getArguments();

if (empty($args)) {
var_dump($GLOBALS['argv']);
$thread = Thread::exec(__FILE__, 'thread-1', $argc, $argv);
$thread->join();
} else {
var_dump($args[0], $args[1], $args[2]);
sleep(1);
}
20 changes: 20 additions & 0 deletions examples/thread/array.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<?php

ini_set('memory_limit', '2G');
$dict = [];
const COUNT = 10000000;

$n = COUNT;
$s = microtime(true);
while ($n--) {
$dict['key-' . $n] = $n * 3;
}
echo 'array set: ' . round(microtime(true) - $s, 6) . ' seconds' . PHP_EOL;

$c = 0;
$n = COUNT;
$s = microtime(true);
while ($n--) {
$c += $dict['key-' . $n];
}
echo 'array get: ' . round(microtime(true) - $s, 6) . ' seconds' . PHP_EOL;
28 changes: 28 additions & 0 deletions examples/thread/atomic.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<?php

use Swoole\Thread;
use Swoole\Thread\Atomic;
use Swoole\Thread\Atomic\Long;

$args = Thread::getArguments();
$c = 4;
$n = 128;

if (empty($args)) {
$threads = [];
$a1 = new Atomic;
$a2 = new Long;
for ($i = 0; $i < $c; $i++) {
$threads[] = Thread::exec(__FILE__, $i, $a1, $a2);
}
for ($i = 0; $i < $c; $i++) {
$threads[$i]->join();
}
var_dump($a1->get(), $a2->get());
} else {
$a1 = $args[1];
$a2 = $args[2];

$a1->add(3);
$a2->add(7);
}
Loading
Loading