Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms

We study the existence of multiple solutions for a quasilinear elliptic system. Based on the Ambrosetti–Rabinowitz mountain-pass theorem and the Rabinowitz symmetric mountain-pass theorem, we establish several existence and multiplicity results for the solutions and G-symmetric solutions under certa...

Full description

Saved in:
Bibliographic Details
Published in:Український математичний журнал
Date:2015
Main Author: Nyamoradi, N.
Format: Article
Language:English
Published: Інститут математики НАН України 2015
Subjects:
Online Access:https://nasplib.isofts.kiev.ua/handle/123456789/165672
Tags: Add Tag
No Tags, Be the first to tag this record!
Journal Title:Digital Library of Periodicals of National Academy of Sciences of Ukraine
Cite this:Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms / N. Nyamoradi // Український математичний журнал. — 2015. — Т. 67, № 6. — С. 788–808. — Бібліогр.: 17 назв. — англ.

Institution

Digital Library of Periodicals of National Academy of Sciences of Ukraine
id nasplib_isofts_kiev_ua-123456789-165672
record_format dspace
spelling Nyamoradi, N.
2020-02-15T16:48:12Z
2020-02-15T16:48:12Z
2015
Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms / N. Nyamoradi // Український математичний журнал. — 2015. — Т. 67, № 6. — С. 788–808. — Бібліогр.: 17 назв. — англ.
1027-3190
https://nasplib.isofts.kiev.ua/handle/123456789/165672
517.9
We study the existence of multiple solutions for a quasilinear elliptic system. Based on the Ambrosetti–Rabinowitz mountain-pass theorem and the Rabinowitz symmetric mountain-pass theorem, we establish several existence and multiplicity results for the solutions and G-symmetric solutions under certain suitable conditions.
Вивчається задача існування багатьох розв'язків квазілінійної еліптичної системи. На основі теореми перевалу Амброзетті і Рабіновича та симетричної теореми перевалу Рабіновича встановлено кілька результатів про існування та множинність розв'язків та G-симетричних розв'язків за деяких прийнятних умов.
en
Інститут математики НАН України
Український математичний журнал
Статті
Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
Розв'язки для квазілінійних еліптичних систем з комбiнованими критичними членами Соболєва-Харді
Article
published earlier
institution Digital Library of Periodicals of National Academy of Sciences of Ukraine
collection DSpace DC
title Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
spellingShingle Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
Nyamoradi, N.
Статті
title_short Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
title_full Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
title_fullStr Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
title_full_unstemmed Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms
title_sort solutions of the quasilinear elliptic systems with combined critical sobolev–hardy terms
author Nyamoradi, N.
author_facet Nyamoradi, N.
topic Статті
topic_facet Статті
publishDate 2015
language English
container_title Український математичний журнал
publisher Інститут математики НАН України
format Article
title_alt Розв'язки для квазілінійних еліптичних систем з комбiнованими критичними членами Соболєва-Харді
description We study the existence of multiple solutions for a quasilinear elliptic system. Based on the Ambrosetti–Rabinowitz mountain-pass theorem and the Rabinowitz symmetric mountain-pass theorem, we establish several existence and multiplicity results for the solutions and G-symmetric solutions under certain suitable conditions. Вивчається задача існування багатьох розв'язків квазілінійної еліптичної системи. На основі теореми перевалу Амброзетті і Рабіновича та симетричної теореми перевалу Рабіновича встановлено кілька результатів про існування та множинність розв'язків та G-симетричних розв'язків за деяких прийнятних умов.
issn 1027-3190
url https://nasplib.isofts.kiev.ua/handle/123456789/165672
citation_txt Solutions of the Quasilinear Elliptic Systems with Combined Critical Sobolev–Hardy Terms / N. Nyamoradi // Український математичний журнал. — 2015. — Т. 67, № 6. — С. 788–808. — Бібліогр.: 17 назв. — англ.
work_keys_str_mv AT nyamoradin solutionsofthequasilinearellipticsystemswithcombinedcriticalsobolevhardyterms
AT nyamoradin rozvâzkidlâkvazílíníinihelíptičnihsistemzkombinovanimikritičnimičlenamisobolêvahardí
first_indexed 2025-11-24T06:40:43Z
last_indexed 2025-11-24T06:40:43Z
_version_ 1850446231584636928
fulltext UDC 517.9 N. Nyamoradi (Razi Univ., Iran) SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL SOBOLEV – HARDY TERMS РОЗВ’ЯЗКИ ДЛЯ КВАЗIЛIНIЙНИХ ЕЛIПТИЧНИХ СИСТЕМ З КОМБIНОВАНИМИ КРИТИЧНИМИ ЧЛЕНАМИ СОБОЛЄВА – ХАРДI We study the existence of multiple solutions for a quasilinear elliptic system. Based upon the Mountain – Pass theorem of Ambrosetti and Rabinowitz and symmetric Mountain – Pass theorem of Rabinowitz, we establish several existence and multiplicity results for the solutions and G-symmetric solutions under certain suitable conditions. Вивчається задача iснування багатьох розв’язкiв квазiлiнiйної елiптичної системи. На основi теореми перевалу Амброзеттi i Рабiновича та симетричної теореми перевалу Рабiновича встановлено кiлька результатiв про iснування та множиннiсть розв’язкiв та G-симетричних розв’язкiв за деяких прийнятних умов. 1. Introduction. Our purpose in the first part of this paper is to establish the existence of nontrivial solution to the following quasilinear elliptic system: −div(|x|−ap|∇u|p−2∇u)− µ |u|p−2u |x|p(a+1) = = |u|p ∗(a,b1)−2u |x|b1p ∗(a,b1) + α α+ β Q(x) |u|α−2|v|βu |x− x0|cp ∗(a,c) + λh(x) |u|q−2u |x|dp ∗(a,d) , x ∈ Ω, −div(|x|−ap|∇v|p−2∇v)− µ |v|p−2v |x|p(a+1) = = |v|p ∗(a,b2)−2v |x|b2p ∗(a,b2) + β α+ β Q(x) |u|α|v|β−2v |x− x0|cp ∗(a,c) + λh(x) |v|q−2v |x|dp ∗(a,d) , x ∈ Ω, u = v = 0, x ∈ ∂Ω, (1) where 0 ∈ Ω is a bounded domain in R N , N ≥ 3, with smooth boundary ∂Ω, λ > 0 is a parameter, 1 ≤ q < p, 1 < p < N, 0 ≤ µ < µ , ( N − p p− a )p , 0 ≤ a < N − p p ; Q(x) is nonnegative and continuous on Ω satisfying some additional conditions which will be given later, Q(x0) = ‖Q‖∞ for 0 6= x0 6= Ω, h(x) ∈ C(Ω); α, β > 1, α + β = p∗(a, c) , pN N − p(1 + a− c) , p∗(a, b1) , , pN N − p(1 + a− b1) (a ≤ b1, b2, d ≤ c < a+ 1) are critical Sobolev – Hardy exponents. Note that p∗(0, 0) = p∗ := Np N − p is the critical Sobolev exponent. In the second part of this paper, we consider the following quasilinear elliptic system: c© N. NYAMORADI, 2015 788 ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 789 −div(|x|−ap|∇u|p−2∇u)− µ |u|p−2u |x|p(a+1) = = α α+ β Q(x) |u|α−2|v|βu |x− x0|cp ∗(a,c) + λh(x) |u|q−2u |x|dp∗(a,d) , x ∈ Ω, −div(|x|−ap|∇v|p−2∇v)− µ |v|p−2v |x|p(a+1) = = β α+ β Q(x) |u|α|v|β−2v |x− x0|cp ∗(a,c) + λh(x) |v|q−2v |x|dp∗(a,d) , x ∈ Ω, u = v = 0, x ∈ ∂Ω, (2) where 0 ∈ Ω is a bounded domain, G-symmetric domain (see Section 4 for details) in R N , N ≥ 3, with smooth boundary ∂Ω, λ > 0 is a parameter, 1 < q < p < p∗(a, c), 1 < p < N, 0 ≤ µ < µ , , ( N − p p− a )p , 0 ≤ a < N − p p and a ≤ d ≤ c < a+ 1. The aim this part (second part) is to establish few results on the existence of G-symmetric solutions for (2). In this paper, if 1 < p < N and −∞ < a < N − p p we denote by W 1,p a (Ω, |x|−ap) the completion of C∞ 0 (Ω) with respect to the norm ‖u‖ =   ∫ Ω |x|−ap|∇u|pdx   1 p . Problem (1) is related to the well known Caffarelli – Kohn – Nirenberg inequality in [1, 2],   ∫ RN |x|−bp∗(a,b)|u|p ∗(a,b)dx   p p∗(a,b) ≤ Ca,b ∫ RN |x|−ap|∇u|pdx for all u ∈ C∞ 0 (RN ), (3) where 1 < p < N, −∞ < a < N − p p , a ≤ b ≤ a+ 1, p∗(a, b) = Np N − p(1 + a− b) . If b = a+ 1, then p∗(a, b) = p and the following Hardy inequality holds [1, 3]: ∫ RN |x|−p(a+1)|u|pdx ≤ 1 µ ∫ RN |x|−ap|∇u|pdx for all u ∈ C∞ 0 (RN ), (4) where µ , ( N − p p− a )p is the best Hardy constant. In the space W 1,p a (Ω, |x|−ap), we employ the following norm if µ < µ: ‖u‖µ = ‖u‖ W 1,p a (Ω,|x|−ap) =   ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) ) dx   1 p . ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 790 N. NYAMORADI By (4) it is equivalent to the usual norm (∫ Ω |x|−ap|∇u|pdx )1/p of the space W 1,p a (Ω, |x|−ap). Now, we define the space W =W 1,p a (Ω, |x|−ap)×W 1,p a (Ω, |x|−ap) with the norm ‖(u, v)‖p = ‖u‖pµ + ‖v‖pµ. Also, we can define the best Sobolev – Hardy constant Sµ,a,b(Ω) = inf u∈W 1,p a (Ω,|x|−ap)\{0} ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) ) dx (∫ Ω |x|−bp∗(a,b)|u|p ∗(a,b)dx ) p p∗(a,b) . (5) From Kang [4] (Lemma 2.2), Sµ,a,b(Ω) is independent of Ω ⊂ R N . Thus, we will simply denote that Sµ,a,b(R N ) = Sµ,a,b(Ω) = Sµ,a,b. For any 0 ≤ µ < µ, α, β > 1 and α + β = p∗(a, c), by (3), (4), 0 ≤ t < p and the Young inequality, the following best constant are well defined: Sµ,α,β,a,c := inf (u,v)∈W\{(0,0)} ∫ Ω ( |x|−ap(|∇u|p + |∇v|p)− µ |u|p + |v|p |x|p(a+1) ) dx (∫ Ω |u|α|v|β |x|cp ∗(a,c) dx ) p p∗(a,b) . (6) Then we have (its proof is the same as that of Theorem 5 in [5]) Sµ,α,β,a,c(µ) =   ( α β ) β α+β + ( β α ) α α+β  Sµ,a,c. Throughout this paper, let R0 be the positive constant such that Ω ⊂ B(0;R0), where B(0;R0) = = {x ∈ R N : |x| < R0}. By Hölder and Sobolev – Hardy inequalities, for all u ∈W 1,p 0 (Ω), we obtain ∫ Ω |u|q |x|dp ∗(a,d) ≤   ∫ B(0;R0) |x|−dp∗(a,d)   p∗(a,d)−q p∗(a,d)   ∫ Ω |u|p ∗(a,d) |x|dp ∗(a,d)   q p∗(a,d) ≤ ≤  NωN R0∫ 0 r−dp∗(a,d)+N−1dr   p∗(a,d)−q p∗(a,d) (Sµ,a,d) − q p ‖u‖q ≤ ≤ D0(Sµ,a,d) − q p ‖u‖q, (7) where ωN = 2πN/2 NΓ ( N 2 ) is the volume of the unit ball in R N and D0 := ( NωNR N−dp∗(a,d) 0 N − dp∗(a, d) )p∗(a,d)−q p∗(a,d) . ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 791 Existence of nontrivial nonnegative solutions for elliptic equations with singular potentials were recently studied by several authors, but, essentially, only with a solely critical exponent. We refer to [6 – 13] and the references therein. For example, in [12] the author studied the following equation via the Mountain – Pass theorem: −div ( |Du|p−2Du |x|ap ) − µ |u|p−2u |x|(a+1)p = |u|p ∗(b)−2u |x|bp∗ + |u|p ∗(c)−2u |x|cp∗ in R N , where 1 < p < N, 0 ≤ µ < µ , ( N−(a+1)p p )p , 0 ≤ a < N − p p , a ≤ b, c < a + 1, p∗(b) = = Np N − (a+ 1− b)p and p∗(c) = Np N − (a+ 1− c)p . In [6], Deng and Huang studied the following quasilinear elliptic problem: −div ( |∇u|p−2∇u |x|ap ) − µ |u|p−2u |x|(a+1)p = Q(x) |u|p ∗(a,b)−2u |x|bp∗(a,b) + h(x, u), x ∈ Ω, u = 0, x ∈ ∂Ω, (8) where Ω ⊂ R N is a smooth bounded domain, 0 ∈ Ω and Ω is G-symmetric with respect to a subgroup G of O(N), Q, h satisfying some suitable conditions and obtained the existence of solutions via variational methods. In this work, motivated by the above works we are interested to study the problems (1) and (2) by using the Mountain – Pass theorem of Ambrosetti and Rabinowitz and symmetric Mountain – Pass theorem of Rabinowitz [14], respectively. We shall show that the system (1) has at least two positive weak solutions and the system (2) has infinitely many G-symmetric solutions. Throughout this paper, we assume that a ≤ b1, b2, d ≤ c < a+1, α, β > 1 and α+β = p∗(a, c). For 0 ≤ µ < µ, we set θ(µ, a, b1) := p∗(a, b1)− p pp∗(a, b1) (Sµ,a,b1) p∗(a,b1) p∗(a,b1)−p , Υ(µ, α, β, a, c) := p∗(a, c) − p pp∗(a, c) 1 ‖Q‖ N−p(a+1−c) p(a+1−c) ∞ (Sµ,α,β,a,c) p∗(a,c) p∗(a,c)−p , θ∗ := { θ(µ, a, b1), θ(µ, a, b2),Υ(µ, α, β, a, c) } . Moreover, assume that Q(x) satisfies some of the following assumptions: (H1) Q ∈ C(Ω), Q(x) ≥ 0 and meas({x ∈ Ω, Q(x) > 0}) > 0. (H2) There exist ϑ > 0 such that Q(x0) = ‖Q‖∞ > 0 and Q(x) = Q(x0) + O(|x − x0| ̺), as x→ x0. (H3) There exist β0 and ρ > 0 such that B2ρ0(x0) ⊂ Ω and h(x) ≥ β0 for all x ∈ B2ρ0(x0). Set h+ := max{h, 0} and h− := max{−h, 0}. The main results of this paper can be included in the following three theorems. Theorem 1. Assume that N ≥ 3, µ ∈ [0, µ), 1 < q < p and (H1). Then there exists Λ∗ 11 > 0, such that for 0 < λ < Λ∗ 11 problem (1) has at lest one positive solutions. ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 792 N. NYAMORADI Theorem 2. Assume that N ≥ p2, 0 ≤ µ < µ, θ∗ = p∗(a, c) − p pp∗(a, c) 1 ‖Q‖ N−p(a+1−c) N ∞ (Sµ,α,β,a,c) p∗(a,c) p∗(a,c)−p , (H1) – (H3), Q(0) = 0, ̺ > b(µ)p + p − N + t and N − dp∗(a, d) β(µ) < q < p hold, and b(µ) is the constant defined as in Lemma 4. Then there exists Λ∗∗ > 0, such that for 0 < λ < Λ∗∗, problem (1) has at least two positive solutions. Theorem 3. Suppose that |G| = +∞ and Q,h ∈ C(Ω) ⋂ L∞(Ω) is G-symmetric. Then for λ > 0 the problem (2) has infinitely many G-symmetric solutions. This paper is divided into four sections, organized as follows. In Section 2, we establish some elementary results. In Section 3, we prove our main results (Theorems 1 and 2). In Section 4, we prove another our main result (Theorem 3). 2. Preliminary lemmas. The corresponding energy functional of problem (1) is defined by J(u, v) = 1 p ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) + |x|−ap|∇v|p − µ |v|p |x|p(a+1) ) dx− − λ q ∫ Ω h(x) ( |u|q |x|dp∗(a,d) + |v|q |x|dp∗(a,d) ) dx− 1 p∗(a, b1) ∫ Ω |u|p ∗(a,b1) |x|b1p∗(a,b1) dx− − 1 p∗(a, b2) ∫ Ω |v|p ∗(a,b2) |x|b2p∗(a,b2) dx− 1 α+ β ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx, for each (u, v) ∈W. Then J ∈ C1(W,R). Lemma 1. Assume that N ≥ 3, 0 ≤ µ < µ, (H1), h+ 6= 0 and (u, v) is a weak solution of problem (1). Then there exists a positive constant d depending on N, |Ω|, |h+|∞, Aµ,s, s1, s2 and q such that J(u, v) ≥ −dλ p p−q . Proof. Since (u, v) is a weak solution of problem (1). Then, note that 〈J ′(u, v), (u, v)〉 = 0, we have 〈J ′(u, v), (u, v)〉 = ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) + |x|−ap|∇v|p − µ |v|p |x|p(a+1) ) dx− −λ ∫ Ω h(x) ( |u|q |x|dp∗(a,d) + |v|q |x|dp∗(a,d) ) dx− ∫ Ω |u|p ∗(a,b1) |x|b1p∗(a,b1) dx− − ∫ Ω |v|p ∗(a,b2) |x|b2p ∗(a,b2) dx− ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx = 0. (9) ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 793 Now, by using h+ 6= 0, (9), (7), the Hölder inequality and the Sobolev – Hardy inequality, by a direct calculation, one can get J(u, v) ≥ 2 inf t≥0 [( 1 p − 1 p∗(a, c) ) tp − λ ( 1 q − 1 p∗(a, c) ) D0(Sµ,a,d) − q p |h+|∞t q ] ≥ −dλ p p−q . Here dΩ := supx,y∈Ω |x− y| is the diameter of Ω and d is a positive constant depending on N, |Ω|, |h+|∞, Sµ,a,d, b1, b2 and q. Lemma 1 is proved. Recall that a sequence (un, vn)n∈N is a (PS)c sequence for the functional J if J(un, vn) → c and J ′(un, vn) → 0. If any (PS)c sequence (un, vn)n∈N has a convergent subsequence, we say that J satisfies the (PS)c condition. Lemma 2. Assume that N ≥ 3, 0 ≤ µ < µ, (H1), h+ 6= 0 and Q(0) = 0. Then J(u, v) satisfies the (PS)c condition with c satisfying c < c∗ := min θ∗ − dλ p p−q . (10) Proof. It is easy to see that the (PS)c sequence (un, vn)n∈N of J(u, v) is bounded in W. Then (un, vn) ⇀ (u, v) weakly in W as n → ∞, which implies un ⇀ u weakly and vn ⇀ v weakly in W 1,p 0 (Ω) as n→ ∞. Passing to a subsequence we may assume that |x|−ap|∇un| pdx ⇀ α, |x|−ap|∇vn| pdx ⇀ α̃, |u|p |x|p(a+1) dx ⇀ β, |v|p |x|p(a+1) dx ⇀ β̃, |un| p∗(a,b1) |x|b1p∗(a,b1) dx ⇀ γ, |vn| p∗(a,b2) |x|b2p∗(a,b2) dx ⇀ γ̃, Q(x) |un| α|vn| β |x− x0|cp ∗(a,c) dx ⇀ ν weakly in the sense of measures. Using the concentration-compactness principle in [15], there exist an at most countable set I, a set of points {xi}i∈I ∈ Ω \ {0}, real numbers axi , ãxi , dxi , i ∈ I, a0, ã0, b0, b̃0, c0, c̃0 and d0, such that α ≥ |x|−ap|∇u|pdx+ ∑ i∈I axi δxi + a0δ0, α̃ ≥ |x|−ap|∇v|pdx+ ∑ i∈I ãxi δxi + ã0δ0, (11) β = |u|p |x|p(a+1) dx+ b0δ0, β̃ = |v|p |x|p(a+1) dx+ b̃0δ0, (12) γ = |u|p ∗(a,b1) |x|b1p∗(a,b1) + c0δ0, γ̃ = |v|p ∗(a,b2) |x|b2p∗(a,b2) dx+ c̃0δ0, (13) ν = Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx+ ∑ i∈I Q(xi)dxi δxi +Q(0)d0δ0, (14) where δx is the Dirac-mass of mass 1 concentrated at the point x. ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 794 N. NYAMORADI First, we consider the possibility of the concentration at {xi}i∈I ∈ Ω \ {0}. Let ǫ > 0 be small enough, take ηxi ∈ C∞ c (B2ε(xi)), such that ηxi |Bε(xi) = 1, 0 ≤ ηxi ≤ 1 and |∇ηxi (x)| ≤ C ε . Then o(1) = 〈J ′(un, vn), (η p xi un, η p xi vn)〉 = = ∫ Ω ( |x|−ap|∇un| p−2∇un∇(ηpxi un) + |x|−ap|∇vn| p−2∇vn∇(ηpxi vn) ) dx− − ∫ Ω Q(x) |un| α|vn| β |x− x0|cp ∗(a,c) ηpxi dx− µ ∫ Ω ( |un| p |x|p(a+1) ηpxi + |vn| p |x|p(a+1) ηpxi ) dx ︸ ︷︷ ︸ (I) − −λ ∫ Ω h(x) ( |un| q |x|dp ∗(a,d) ηpxi + |vn| q |x|dp ∗(a,d) ηpxi ) dx ︸ ︷︷ ︸ (II) −   ∫ Ω |un| p∗(a,b1) |x|b1p ∗(a,b1) ηpxi dx+ ∫ Ω |vn| p∗(a,b2) |x|b2p ∗(a,b2) ηpxi dx   ︸ ︷︷ ︸ (III) . From (12) – (14), one can get lim ε→0 lim n→∞ (I) = lim ε→0   ∫ Ω ηpxi dβ + ∫ Ω ηpxi dβ̃   = 0, lim ε→0 lim n→∞ (III) = lim ε→0   ∫ Ω ηpxi dγ + ∫ Ω ηpxi dγ̃   = 0, lim ε→0 lim n→∞ (II) = 0 (15) and lim ε→0 lim n→∞ ∫ Ω Q(x) |un| α|vn| β |x− x0|cp ∗(a,c) ηpxi dx = lim ε→0 ∫ Ω ηpxi dν = Q(xi)dxi. Thus, 0 = lim ε→0 lim n→∞ ∫ Ω ( |x|−ap|∇un| p−2∇un∇(ηpxi un) + |x|−ap|∇vn| p−2∇vn∇(ηpxi vn) ) dx−Q(xi)dxi. (16) Moreover, by a direct calculation, we have lim ε→0 lim n→∞ ∣∣∣∣∣∣ ∫ Ω |x|−apun|∇un| p−2∇un∇η p xi dx ∣∣∣∣∣∣ = ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 795 = lim ε→0 lim n→∞ ∣∣∣∣∣∣ ∫ Ω |x|−a(p−1)|x|−aun|∇un| p−2∇un∇η p xi dx ∣∣∣∣∣∣ ≤ ≤ C lim ε→0   ∫ Bε(xi) |x|−ap∗ |u|p ∗ dx   p p∗ = 0. (17) Similarly, lim ε→0 lim n→∞ ∣∣∣∣∣∣ ∫ Ω |x|−apvn|∇vn| p−2∇vn∇η p xi dx ∣∣∣∣∣∣ = 0. (18) Combining (16) – (18), there holds 0 = lim ε→0 lim n→∞ ∫ Ω (|x|−ap|ηxi ∇un| p + |x|−ap|ηxi ∇vn| p)dx−Q(xi)dxi = = lim ε→0 ∫ Ω ( ηpxi dα+ ηxi dα̃ ) −Q(xi)dxi . (19) On the other hand, (6) implies that 1 ‖Q‖ p p∗(a,c) ∞ Sµ,α,β,a,c   ∫ Ω Q(x) |ηxi un| α|ηxi vn| β |x− x0|cp ∗(a,c) dx   p p∗(a,c) ≤ ≤ ∫ Ω ( |x|−ap|∇(ηxi un)| p + |x|−ap|∇(ηxi vn)| p − µ |ηxi un| p + |ηxi vn| p |x|p(a+1) ) dx. (20) Note that lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇ηxi |p|un| pdx = lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇ηxi |p|vn| pdx = 0, together with (17) and (18), we obtain that lim ε→0 lim n→∞ ∫ Ω |x|−ap|ηxi ∇un| pdx = lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇(ηxi un)| pdx, (21) lim ε→0 lim n→∞ ∫ Ω |x|−ap|ηxi ∇vn| pdx = lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇(ηxi vn)| pdx. (22) The relations (14), (15) and (20) – (22) imply that ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 796 N. NYAMORADI 1 ‖Q‖ p p∗(a,c) ∞ Sµ,α,β,a,c (Q(xi)dxi ) p p∗(a,c) ≤ lim ε→0 ∫ Ω |ηxi |pdα+ lim ε→0 ∫ Ω |ηxi |pdα̃. (23) Combining (19) and (23), 1 ‖Q‖ p p∗(a,c) ∞ Sµ,α,β,a,c (Q(xi)dxi ) p p∗(a,c) ≤ Q(xi)dxi , (24) which implies that either Q(xi)dxi = 0, or Q(xi)dxi ≥ 1 ‖Q‖ N−p(a+1−c) p(a+1−c) ∞ (Sµ,α,β,a,c) N p(a+1−c) . (25) Now, we consider the possibility of the concentration at 0. For ǫ > 0 be small enough, take η0 ∈ C∞ c (B2ε(0)), such that η0|Bε(0) = 1, 0 ≤ η0 ≤ 1 and |∇η0(x)| ≤ C ε . Then o(1) = 〈J ′(un, vn), (η p 0un, 0)〉 = = ∫ Ω |x|−ap|∇un| p−2∇un∇(ηp0un)dx− µ ∫ Ω |un| p |x|p(a+1) η p 0dx− λ ∫ Ω h(x) |un| q |x|dp∗(a,d) η p 0dx− − ∫ Ω |un| p∗(a,b1) |x|b1p∗(a,b1) η p 0dx− α α+ β ∫ Ω Q(x) |un| α|vn| β |x− x0|cp ∗(a,c) η p 0dx. From (12) – (14) and Q(0) = 0, we obtain that lim ε→0 lim n→∞ ∫ Ω |un| p |x|p(a+1) η p 0dx = b0, lim ε→0 lim n→∞ ∫ Ω |un| p∗(a,b1) |x|b1p∗(a,b1) η p 0dx = c0 and lim ε→0 lim n→∞ ∫ Ω Q(x) |un| α|vn| β |x− x0|cp ∗(a,c) η p 0dx = lim ε→0 lim n→∞ ∫ Ω h(x) |un| q |x|dp ∗(a,d) η p 0dx = 0. Thus, 0 = lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇un| p−2∇un∇(ηp0un)dx− µb0 − c0. (26) Note that lim ε→0 lim n→∞ ∫ Ω |x|−apun|∇un| p−2∇un∇η p 0dx = 0, ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 797 together with (26), there holds lim ε→0 ∫ Ω η p 0dα− µb0 = c0. (27) On the other hand, (5) implies that Sµ,a,b1   ∫ Ω |η0un| p∗(a,b1) |x|b1p∗(a,b1) dx   p p∗(a,b1) ≤ ∫ Ω ( |x|−ap|∇(η0un)| p − µ |η0un| p |x|p(a+1) ) dx. Thus Sµ,a,b1c p p∗(a,b1) 0 ≤ lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇(η0un)| pdx− µb0. (28) Note that lim ε→0 lim n→∞ ∫ Ω |x|−ap|η0∇un| pdx = lim ε→0 lim n→∞ ∫ Ω |x|−ap|∇(η0un)| pdx, together with (28), there holds Sµ,a,b1c p p∗(a,b1) 0 ≤ lim ε→0 ∫ Ω |η0| pdα− µb0. (29) Therefore, from (27) and (29), Sµ,a,b1c p p∗(a,b1) 0 ≤ c0, (30) which implies that either c0 = 0, or c0 ≥ S N p(a+1−b1) µ,a,b1 , (31) similarly, either c0 = 0, or c0 ≥ S N p(a+1−b2) µ,a,b2 . (32) Recall that un ⇀ u weakly and vn ⇀ v weakly in W 1,p a (Ω, |x|−ap), we have c+ o(1) = J(un, vn) = = 1 p ∫ Ω ( |x|−ap|∇un −∇u|p − µ |un − u|p |x|p(a+1) + |x|−ap|∇vn −∇v|p − µ |vn − v|p |x|p(a+1) ) dx− − 1 p∗(a, b1) ∫ Ω |un − u|p ∗(a,b1) |x|b1p ∗(a,b1) dx− 1 p∗(a, b2) ∫ Ω |vn − v|p ∗(a,b2) |x|b2p ∗(a,b2) dx− ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 798 N. NYAMORADI − 1 p∗(a, c) ∫ Ω Q(x) |un − u|α|vn − v|β |x− x0|cp ∗(a,c) dx+ J(u, v). (33) On the other hand, from o(1) = J ′(un, vn), we obtain that J ′(un, vn) = 0. (34) Thus, 0 = 〈J ′(u, v), (u, v)〉. Together with o(1) = 〈J ′(un, vn), (un, vn)〉, there holds o(1) = ∫ Ω ( |x|−ap|∇un −∇u|p − µ |un − u|p |x|p(a+1) + |x|−ap|∇vn −∇v|p − µ |vn − v|p |x|p(a+1) ) dx− − ∫ Ω |un − u|p ∗(a,b1) |x|b1p∗(a,b1) dx− ∫ Ω |vn − v|p ∗(a,b2) |x|b2p∗(a,b2) dx− ∫ Ω Q(x) |un − u|α|vn − v|β |x− x0|cp ∗(a,c) dx. (35) From (33) – (35) and Lemma 1, c+ o(1) ≥ ( 1 p − 1 p∗(a, b1) )∫ Ω |un − u|p ∗(a,b1) |x|b1p∗(a,b1) dx+ ( 1 p − 1 p∗(a, b2) )∫ Ω |vn − v|p ∗(a,b2) |x|b2p∗(a,b2) dx+ + ( 1 p − 1 p∗(a, c) )∫ Ω Q(x) |un − u|α|vn − v|β |x− x0|cp ∗(a,c) dx− dλ p p−q . (36) Passing to the limit in (36) as n→ ∞, we have c ≥ ( 1 p − 1 p∗(a, b1) ) c0 + ( 1 p − 1 p∗(a, b2) ) c̃0 + ( 1 p − 1 p∗(a, c) )∑ i∈I Q(xi)dxi − dλ p p−q . (37) By the assumption c < c∗ and in view of (25), (31) and (32), there holds c0 = c̃0 = 0, Q(xi)dxi = 0, i ∈ I. Up to a subsequence, (un, vn) → (u, v) strongly in W as n→ ∞. Lemma 2 is proved. If the restriction Q(0) = 0 is removed, we establish the following version of Lemma 2. Lemma 3. Assume that N ≥ 3, 0 ≤ µ < µ, (H1) and h+ 6= 0. Then J(u, v) satisfies the (PS)c condition with c satisfying c < c0 := min { p∗(a, b1)− p pp∗(a, b1) ( 1 p Sµ,a,b1 ) p∗(a,b1) p∗(a,b1)−p , p∗(a, b2)− p pp∗(a, b2) ( 1 p Sµ,a,b2 ) p∗(a,b2) p∗(a,b2)−p , p∗(a, c) − p pp∗(a, c) 1 ‖Q‖ N−p(a+1−c) N ∞ ( 1 p Sµ,α,β,a,c ) p∗(a,c) p∗(a,c)−p } − dλ p p−q . (38) Proof. The proof is similar to Lemma 2 and is omitted. Here, we recall a recent result on the extremal functions of Sµ,a,b [4]. ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 799 Lemma 4 [4]. Assume that 0 ≤ a < N − p p , a ≤ b < a + 1 and 0 ≤ µ < µ. Then Sµ,a,b is attained when Ω = R N by the radial functions Vǫ(x) , ǫ −(N−p p −a) Up,µ ( |x| ǫ ) ∀ǫ > 0, (39) that satisfy ∫ Ω ( |x|−ap|∇Vǫ(x)| p − µ |Vǫ(x)| p |x|p(a+1) ) dx = ∫ Ω |Vǫ(x)| p∗(a,b) |x|bp∗(a,c) dx = (Sµ,a,c) p∗(a,c) p∗(a,c)−p , where Up,µ(x) = Up,µ(|x|) is the unique radial solution of the following problem: −div(|x|−ap|∇u|p−2∇u)− µ |u|p−2u |x|p(a+1) = |u|p ∗(a,c)−2u p∗(a, c)|x|cp∗(a,c) , in R N \ {0}, u ∈W 1,p a (RN ), u > 0, in R N \ {0}, with Up,µ(1) = ( p∗(a, c)(µ − µ) p ) 1 p∗(a,c)−p . Furthermore, Up,µ have the following properties: lim r→0 rα(µ)Up,µ(r) = C1 > 0, lim r→+∞ rβ(µ)Up,µ(r) = C2 > 0, lim r→0 rα(µ)+1|U ′ p,µ(r)| = C1α(µ) ≥ 0, lim r→+∞ rβ(µ)+1|U ′ p,µ(r)| = C2β(µ) > 0, where Ci, i = 1, 2, are positive constants and α(µ) and β(µ) are zeros of the function f(ζ) = (p− 1)ζp − (N − p(a+ 1))ζp−1 + µ, ζ ≥ 0, 0 ≤ µ < µ, that satisfy 0 ≤ α(µ) < N − p(a+ 1) p < β(µ) ≤ N − p(a+ 1) p− 1 . Furthermore, there exist the positive constants C3 = C3(µ, p, a, c) and C4 = C4(µ, p, a, c) such that C3 ≤ Up,µ(x) ( |x| α(µ) δ − |x| β(µ) δ )δ < C4, δ = N − p(a+ 1) p . Lemma 5. Under the assumptions of Theorem 2, there exists (u1, v1) ∈W \{(0, 0)} and Λ1 > 0, such that for 0 < λ < Λ1, there holds sup t≥0 J(tu1, tv1) < Υ(µ, α, β, a, c) − dλ p p−q . (40) ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 800 N. NYAMORADI Proof. First, we will give some estimates on the extremal function Vǫ(x) defined in (39). Let Vǫ(x) be the function in Lemma 4, ρ > 0 small enough such that Bρ(0) ⊂ Ω, ψ ∈ C∞ 0 (Bρ(0)) with 0 ≤ ψ ≤ 1 in Bρ(0) and ψ = 1 in Bρ/2(0), then the function given by [4]: uǫ(x) := ψ(x)Vǫ(x), satisfies ‖uǫ‖p = (Sµ,a,c) p∗(a,c) p∗(a,c)−p +O ( ǫβ(µ)p+p(a+1)−N ) , (41) ∫ Ω |uǫ| p∗(a,c) |x|cp ∗(a,c) dx = (Sµ,a,c) p∗(a,c) p∗(a,c)−p +O ( ǫ(β(µ)+c)p∗(a,c)−N ) , (42) ∫ Ω |uǫ| q |x|dp∗(a,d) dx ≥    CǫN−dp∗(a,d)−qδ , if N − dp∗(a, d) β(µ) < q < p∗(a, d), Cǫq(β(µ)−δ)| ln(ǫ)|), if q = N − dp∗(a, d) β(µ) , Cǫq(β(µ)−δ), if 1 ≤ q < N − dp∗(a, d) β(µ) , (43) where δ = N − p(a+ 1) p < β(µ) ≤ N − p(a+ 1) p− 1 . Now, we consider the functional I : W → R defined by I(u, v) = 1 p ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) + |x|−ap|∇v|p − µ |v|p |x|p(a+1) ) dx− − 1 p∗(a, c) ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx. Let u1 = α1/puǫ, v1 = β1/puǫ and define the function g1(t) := J(tu1, tv1), t ≥ 0. Note that limt→+∞ g1(t) = −∞ and g1(t) > 0 as t is close to 0. Thus supt≥0 g1(t) is attained at some finite tǫ > 0 with g′1(tǫ) = 0. Furthermore, C ′ < tǫ < C ′′; where C ′ and C ′′ are the positive constants independent of ǫ. We have I(tu1, tv1) = [ tp p (α+ β) ∫ Ω ( |x|−ap|∇uǫ| p − µ |uǫ| p |x|p(a+1) ) dx− − tp ∗(a,c) p∗(a, c) (αα/pββ/p) ∫ Ω Q(y0) |uǫ| p∗(a,c) |x− x0|p ∗(a,c)c dx ] − − tp ∗(a,c) p∗(a, c) (αα/pββ/p) ∫ Ω (Q(x)−Q(x0)) |uǫ| p∗(a,c) |x− x0|cp ∗(a,c) dx := ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 801 := y(tu1, tv1)− tp ∗(a,c) p∗(a, c) (αα/pββ/p) ∫ Ω (Q(x)−Q(x0)) |uǫ| p∗(a,c) |x− x0|cp ∗(a,c) dx. (44) Note that sup t≥0 ( tp p A− tp ∗(a,c) p∗(a, c) B ) = ( 1 p − 1 p∗(a, c) )( A B p p∗(a,c) ) p∗(a,c) p∗(a,c)−p , A,B > 0. (45) From (H2), (41), (42) and (45) it follows that straightforward sup t≥0 y(tu1, tv1) ≤ ( 1 p − 1 p∗(a, c) ) 1 ‖Q‖ N−p(a+1−c) p(a+1−c) ∞ (Sµ,α,β,a,c) p∗(a,c) p∗(a,c)−p +O ( ǫβ(µ)p+p(a+1)−N ) . (46) On the other hand, (H2) implies that there exists r1 < r, such that for x ∈ Br1(y0), |Q(x)−Q(x0)| ≤ ≤ C|x− x0| ϑ. Thus ∣∣∣∣∣∣ ∫ Ω (Q(x)−Q(x0)) |uǫ| p∗(a,c) |x− x0|cp ∗(a,c) dx ∣∣∣∣∣∣ ≤ C ∫ Ω |Q(x)−Q(x0)| |uǫ| p∗(a,c) |x− x0|cp ∗(a,c) dx = = C ∫ B2r(x0) |x− x0| ϑ|uǫ| p∗(a,c) |x− x0|cp ∗(a,c) dx = = O(ǫϑ−cp∗(a,c)). (47) From (44), (46), one can get sup t≥0 I(tu1, tv1) = I(tǫu1, tǫv1) ≤ Υ(µ, α, β, a, c) +O ( ǫβ(µ)p+p(a+1)−N ) . (48) Observe that there exists Λ∗ 1 > 0, such that for 0 < λ < Λ∗ 1 and Υ(µ, α, β, a, c) − dλ p p−q > 0, Then for 0 < λ < Λ∗ 1, there exists t1 ∈ (0, 1), such that sup 0≤t≤t1 J(tu1, tv1) ≤ sup 0≤t≤t1 1 p tp ∫ Ω ( |x|−ap|∇u1| p + |x|−ap|∇v1| p ) dx < Υ(µ, α, β, a, c) − dλ p p−q . (49) On the other hand, we have sup t≥t1 J(tu1, tv1) ≤ sup t≥t1  I(tu1, tv1)− λ q tq ∫ Ω h(x) |u1| q |x|dp ∗(a,d) dx− tp ∗(a,b1) p∗(a, b1) ∫ Ω |u1| p∗(a,b1) |x|b1p ∗(a,b1) dx   ≤ ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 802 N. NYAMORADI ≤ sup t≥t1  I(tu1, tv1)− λ q t q 1 ∫ Ω h(x) |u1| q |x|dp∗(a,d) dx− t p∗(a,b1) 1 p∗(a, b1) ∫ Ω |u1| p∗(a,b1) |x|b1p∗(a,b1) dx   ≤ ≤ Υ(µ, α, β, a, c) +O ( ǫβ(µ)p+p(a+1)−N ) − −C ∫ Ω |uǫ| p∗(a,b1) |x|b1p ∗(a,b1) dx− λC ∫ Ω h(x) |uǫ| q |x|dp ∗(a,d) dx. (50) From (42), ∫ Ω |uǫ| p∗(a,b1) |x|b1p ∗(a,b1) dx ≥ O ( ǫ(β(µ)+b1)p∗(a,b1)−N ) . (51) Also, from (43), it follows that ∫ Ω h(x) |uǫ| q |x|dp ∗(a,d) dx ≥ β0 ∫ Ω |uǫ| q |x|dp ∗(a,d) dx ≥ ≥    CǫN−dp∗(a,d)−qδ , if N − dp∗(a, d) β(µ) < q < p∗(a, d), Cǫq(β(µ)−δ)| ln(ǫ)|), if q = N − dp∗(a, d) β(µ) , Cǫq(β(µ)−δ), if 1 ≤ q < N − dp∗(a, d) β(µ) . (52) Since q ≥ N − dp∗(a, d) β(µ) , by (50) – (52) we have sup t≥t1 J(tu1, tv1) ≤ Υ(µ, α, β, a, c) +O ( ǫβ(µ)p+p(a+1)−N ) +O ( ǫ(β(µ)+b1)p∗(a,b1)−N ) − −λ    CǫN−dp∗(a,d)−qδ , if N − dp∗(a, d) β(µ) < q < p∗(a, d), Cǫq(β(µ)−δ)| ln(ǫ)|), if q = N − dp∗(a, d) β(µ) . Note that β(µ)p+ p(a+ 1)−N < (β(µ) + b1)p ∗(a, b1)−N, then we have sup t≥t1 J(tu1, tv1) ≤ Υ(µ, α, β, a, c) +O ( ǫ(β(µ)+b1)p∗(a,b1)−N ) − −λ    CǫN−dp∗(a,d)−qδ , if N − dp∗(a, d) β(µ) < q < p∗(a, d), Cǫq(β(µ)−δ)| ln(ǫ)|), if q = N − dp∗(a, d) β(µ) . (53) ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 803 Note that N > p2, β(µ) ≥ N − dp∗(a, d) q . Thus [ N − dp∗(a, d) − qδ ]p− q q < β(µ)p + p(a+ 1)−N − [ N − dp∗(a, d) − qδ ] . Choose λ = ǫτ , where [ N−dp∗(a, d)−qδ ]p− q q < τ < β(µ)p+p(a+1)−N− [ N−dp∗(a, d)−qδ ] . Then λO(ǫN−dp∗(a,d)−qδ) = O(ǫτ+N−dp∗(a,d)−qδ) and dλ p p−q = O(ǫ pτ p−q ). Since τ +N − dp∗(a, d)− qδ < pτ p− q , τ +N − dp∗(a, d)− qδ < β(µ)p+ p(a+1)−N, taking ǫ small enough we deduce that there exists δ > 0, such that O ( ǫ(β(µ)+b1)p∗(a,b1)−N ) − λO(ǫN−dp∗(a,d)−qδ) < −dλ p p−q ∀λ : 0 < λ p p−q < δ. (54) Choose Λ1 = min { Λ∗ 1, p− q p δ } . Then for all λ ∈ (0,Λ1) we have sup t≥t1 J(tu1, tv1) ≤ Υ(µ, α, β, a, c) − dλ p p−q . Together with (49), we get the conclusion of Lemma 5. 3. Proof of the main results. Proof of Theorem 1. Let r := ‖(u, v)‖, f(r) := 1 p rp − 1 p∗(a, b1) S − p∗(a,b1) p µ,a,b1 rp ∗(a,b1)− − 1 p∗(a, b2) S − p∗(a,b2) p µ,a,b2 rp ∗(a,b2) − 1 p∗(a, c) S − p∗(a,c) p µ,α,β,a,c ‖Q‖∞, h(r) := D0(Sµ,a,d) − q p rq. From (6) and (7), J(u, v) ≥ f(r)− h(r), Note that p < p∗(a, b1), p ∗(a, b2), p ∗(a, c), it is easy to see that there exists ̺ > 0 such that f(r) achieves its maximum at ̺ and f(̺) > 0. Therefore, there exists Λ11 > 0, such that for 0 < λ < Λ11, inf ‖(u,v)‖=̺ I(u, v) ≥ f(̺)− h(̺) > 0. (55) On the other hand, set B̺ = {(u, v); ‖(u, v)‖ ≤ ̺} . For (u, v) 6= (0, 0), we can choose d > 0 small enough, such that (du, dv) ∈ B̺ and I(du, dv) < 0. (56) ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 804 N. NYAMORADI Thus, −∞ < inf (u,v)∈B̺ I(u, v) < 0. (57) Now we can apply the Ekeland variational principle in [16] and obtain {(un, vn)} ⊂ B̺, such that I(un, vn) ≤ inf (u,v)∈B̺ I(u, v) + 1 n , (58) I(un, vn) ≤ I(u, v) + 1 n ‖(un − u, vn − v)‖, (59) for all (u, v) ∈ BR. Define J1(u, v) := J(u, v) + 1 n ‖(un − u, vn − v)‖. (60) By (59), we have (un, vn) is the minimizer of J1(u, v) on B̺. (55), (57) and (58) imply that there exists ǫ > 0 and k ∈ N, such that for n ≥ k, {(u, v), ‖(u, v)‖ ≤ ̺− ǫ}. Therefore, for n ≥ k and (φ,ϕ) ∈W, we can choose t > 0 small enough, such that (un + tφ, vn + tϕ) ∈ B̺ and J1(un + tφ, vn + tϕ)− J1(un, vn) t ≥ 0. That is, J(un + tφ, vn + tϕ)− J(un, vn) t + 1 n ‖(φ,ϕ)‖ ≥ 0. (61) Passing to the limit in (61) as n→ 0, one can get 〈J ′(un, vn), (φ,ϕ)〉 ≥ − 1 n ‖(φ,ϕ)‖, which implies that ‖J ′(un, vn)‖ ≤ 1 n . (62) Combining (58) and (62), there holds lim n→∞ J(un, vn) = inf (u,v)∈B̺ J(u, v) < 0, (63) lim n→∞ J ′(un, vn) = 0. (64) We note that there exists Λ∗ 11 ∈ (0,Λ11), such that for 0 < λ < Λ∗ 11, and c0 > inf(u,v)∈B̺ I(u, v), where c0 is defined in Lemma 3. Thus, (63) and (64) and Lemma 5 imply that for 0 < λ < < Λ∗ 11, (un, vn) → (u, v) strongly in W. Therefore, (u, v) is a nontrivial solution of problem (1) satisfying J(u, v) = inf(u,v)∈B̺ J(u, v) < 0. Note that J(u, v) = J(|u|, |v|) and (|u|, |v|) ∈ {(u, v), ‖(u, v)‖ ≤ ̺− ǫ} , ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 805 we have I(|u|, |v|) = inf(u,v)∈B̺ J(u, v) and J ′(|u|, |v|) = 0. Then problem (1) has a nontrivial nonnegative solution. By the strongly maximum principle, we get the conclusion of Theorem 1. Proof of Theorem 2. In view of the proof of Theorem 1, we know that for 0 < λ < Λ11, there exists ̺ > 0, such that inf‖(u,v)‖=̺ I(u, v) ≥ ϑ∗ > 0. Moreover, (63) and (64) hold. We note that there exists Λ12 ∈ (0,Λ11), such that for 0 < λ < Λ12, c∗ > inf(u,v)∈B̺ J(u, v), where c∗ is defined in Lemma 2. Thus (63) and (64) and Lemma 2 imply that (un, vn) → (u, v) strongly in W. Standard argument shows that for 0 < λ < Λ12, problem (1) has at least one positive solution satisfying J(u, v) < 0 and J ′(u, v) = 0. Now we prove a second positive solution. It is easy to see J(0, 0) = 0. Set Λ∗∗ = min{Λ12,Λ1}, where Λ1 is given in Lemma 5. Then it follows from Lemma 5 there exists (u′, v′) ∈W \ {0}, such that for 0 < λ < Λ∗∗, sup t≥0 J(tu′, tv′) < c∗. On the other hand we obtain that liml→∞ J(lu′, lv′) = −∞. Thus there exists l′ > 0 such that ‖(l′u′, l′v′)‖ > ̺ and J(l′u′, l′v′) < 0. Let c := inf γ∈Γ sup t∈[0,1] J(γ(t)), where Γ := { γ ∈ C0([0, 1],W ) | γ(0) = (0, 0), γ(1) = (l′u′, l′v′) } . Thus, it follows from the Mountain pass theorem in [14] that there exists a sequence (un, vn) ∈ W such that lim n→∞ J(un, vn) = c and lim n→∞ J ′(un, vn) = 0. Moreover, c ∈ (0, c∗). From Lemma 2, (un, vn) → (u, v) strongly inW, which implies that J(u, v) = = c and J ′(u, v) = 0, Therefore, (u, v) is a second nontrivial solution of problem(1). Set u+ = = max{u, 0}, v+ = max{v, 0}. Replacing ∫ Ω |u|q |x|dp∗(a,d) dx, ∫ Ω |v|q |x|dp∗(a,d) dx, ∫ Ω |u|p ∗(a,b1) |x|b1p∗(a,b1) dx, ∫ Ω |v|p ∗(a,b2) |x|b2p∗(a,b2) dx, ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx by ∫ Ω (u+)q |x|dp∗(a,d) dx, ∫ Ω (v+)q |x|dp∗(a,d) dx, ∫ Ω (u+)p ∗(a,b1) |x|b1p∗(a,b1) dx, ∫ Ω (v+)p ∗(b2) |x|b2p ∗(a,b2) dx, ∫ Ω Q(x) (u+)α(v+)β |x− x0|cp ∗(a,c) dx ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 806 N. NYAMORADI and repeating the above process, we have a nonnegative solution (ũ, ṽ) of problem (1) satisfying J(ũ, ṽ) > 0. Then by the strongly maximum principle, we have a second positive solution. Theorem 2 is proved. 4. Symmetric solution. In this section, similar to the method in [6], we prove the existence of infinitely many G-symmetric solutions of problem (2). The corresponding energy functional of problem (2) is defined by J̃(u, v) = 1 p ∫ Ω ( |x|−ap|∇u|p − µ |u|p |x|p(a+1) + |x|−ap|∇v|p − µ |v|p |x|p(a+1) ) dx− − λ q ∫ Ω h(x) ( |u|q |x|dp∗(a,d) + |v|q |x|dp∗(a,d) ) dx− 1 α+ β ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx. (65) First, we present some notations and definitions that will be used in this section. Let O(N) be the group of orthogonal linear transformations of R N with natural action and let G ⊂ O(N) be a subgroup with the property that Fix{G} = {0}, where Fix{G} = {x ∈ R N : gx = x ∀g ∈ G} is the fixed point set of the action of G on R N . For x 6= 0 we denote the cardinality of Gx = {gx : g ∈ G} by |Gx| and set |G| = inf06=x∈RN |Gx|. Note that, here, |G| may be +∞. We call Ω a G-symmetric subset of RN , if x ∈ Ω, then gx ∈ Ω for all g ∈ G. For any function f(x) defining on R N , We call f(x) a G-symmetric function if for all g ∈ G and x ∈ R N , f(gx) = f(x) holds. In particular, if f is radially symmetric, then the corresponding group G is O(N) and |G| = +∞. Other further examples of G-symmetric functions can be found in [17]. For a bounded and G-symmetric domain Ω ⊂ R N , 0 ∈ Ω, the natural functional space to study problem (2) is the Banach space WG(Ω) = W 1,p a,G(Ω, |x| −ap) × W 1,p a,G(Ω, |x| −ap) which W 1,p a,G(Ω, |x| −ap) is the subspace of W 1,p a (Ω, |x|−ap) consisting of all G-symmetric functions. Lemma 6. Assume that N ≥ 3, 0 ≤ µ < µ, h+ 6= 0 and Q ∈ C(Ω) ⋂ L∞(Ω) is G-symmetric. Then J̃(u, v) satisfies the (PS)c condition in WG with c satisfying c < c∗ := p∗(a, c)− p pp∗(a, c) |G| ‖Q‖ N−p(a+1−c) N ∞ ( 1 p Sµ,α,β,a,c ) p∗(a,c) p∗(a,c)−p . (66) Proof. The proof is similar to the Lemma 3.3 of [6] and the Lemma 3 and is omitted. Corollary 1. If |G| = +∞, then the functional J̃ satisfies (PS)c condition for every c ∈ R. To prove Theorem 3 we need the following version of symmetric mountain pass theorem (see [14], Theorem 9.12). Theorem 4. Let E be an infinite dimensional Banach space and F ∈ C1(E,R) be an even functional satisfying (PS)c condition for each c and F = 0. Further, we suppose that: (i) there exist constants α̃ > 0 and ρ > 0 such that F ≥ α̃ for all ‖u‖ = ρ; (ii) there exist an increasing sequence of subspaces {Em} of E, with dimEm = m, such that for every m one can find a constant Rm > 0 such that F ≤ 0 for all u ∈ Em with ‖u‖ ≥ Rm. Then F possesses a sequence of critical values {cm} tending to ∞ as m→ ∞. ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 SOLUTIONS FOR THE QUASILINEAR ELLIPTIC SYSTEMS WITH COMBINED CRITICAL . . . 807 Proof of Theorem 3. The proof is similar to that of Theorem 2.2 in [6]. From α+β = p∗(a, c), the Young inequality and (5) it follows that ∫ Ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx ≤ α‖Q‖∞ α+ β ∫ Ω |u|α+β |x− x0|cp ∗(a,c) dx+ β‖Q‖∞ α+ β ∫ Ω |v|α+β |x− x0|cp ∗(a,c) dx ≤ ≤ α‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖u‖p ∗(a,c) µ + β‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖v‖p ∗(a,c) µ . (67) So, by (7), (65) and (67), one can get J̃(z) = J̃(u, v) ≥ ≥ 1 p ‖u‖pµ + 1 p ‖v‖pµ − λD0(Sµ,a,d) − q p (‖u‖q + ‖v‖q)− − α‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖u‖p ∗(a,c) µ − β‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖v‖p ∗(a,c) µ . Since 1 < q < p < p∗(a, c), we see that J̃(z) ≥ ‖z‖q [ 1 p ‖z‖p−q − λD0(Sµ,a,d) − q p ] − − α‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖z‖p ∗(a,c) − β‖Q‖∞ p∗(a, c) (Sµ,a,c) − p∗(a,c) p ‖z‖p ∗(a,c). Now, taking ‖z‖ = ρ such that ρp−q = 2pλ ( NωNR N−dp∗(a,d) 0 N − dp∗(a, d) )p∗(a,d)−q p∗(a,d) (Sµ,a,d) −q/p > 0 with λ > 0. Finally, we take β̂ > 0 such that J̃(z) ≥ λD0(Sµ,a,d) − q p ρq − β̂ρp ∗(a,c) − β̂ρp ∗(a,c) > 0, for every z = (u, v) ∈ WG(Ω) and ‖z‖ = ρ. Therefore, there exist α̂ > 0 and ρ > 0 such that J̃(z) ≥ α̂ for every z with ‖z‖ = ρ. On the other hand, to find a suitable sequence of finite dimensional subspaces of WG(Ω), we set ω = {x ∈ Ω;Q(x) > 0}. Since the set ω is G-symmetric, we can define WG(ω), which is the subspace of G-symmetric functions of W |Ω=ω. By extending functions in WG(ω) to 0 outside ω we can assume WG(ω) ⊂ WG(Ω). Let {Em} be an increasing sequence of subspaces of WG(ω) with dimEm = m for each m. Now, we take ϕ1,m, . . . , ϕm,m ∈ C∞ 0 (Ω) such that 0 ≤ ϕi,m ≤ 1, supp(ϕi,m) ⋂ supp(ϕj,m) = ∅, i 6= j, and |supp(ϕi,m) ⋂ ω| > 0, for all i, j ∈ {1, . . . ,m}. Let ei,m = (aϕi,m, bϕi,m) ∈ Em, i = 1, . . . ,m, and Em = span{e1,m, . . . , em,m}, where a, b are tow positive constants. By construction, dimEm = m. Now, for z = (u, v) = ∑m i=1 ti,mei,m ∈ Em, one can get 1 α+ β ∫ ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx = 1 α+ β ∫ ω Q(x) ∣∣∣ ∑m i=1 ati,mϕi,m ∣∣∣ α ∣∣∣ ∑m i=1 bti,mϕi,m ∣∣∣ β |x− x0|cp ∗(a,c) dx, ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6 808 N. NYAMORADI then there exists a constant C(m) > 0 such that 1 α+ β ∫ ω Q(x) |u|α|v|β |x− x0|cp ∗(a,c) dx ≥ c(m), for all (u, v) ∈ Em, with ‖z‖ = ‖(u, v)‖ = 1. Consequently, if 0 6= u ∈ Em, then we write z = (u, v) = (tu1, tv1), with t = ‖z‖ and ‖(u1, v1)‖ = 1. Thus we have J̃(z) ≤ J̃0(u, v) = 1 p tp − tp ∗(a,c) α+ β ∫ ω Q(x) |u1| α|v1| β |x− x0|cp ∗(a,c) dx ≤ 1 p tp − C(m)tp ∗(a,c) ≤ 0 for t large enough. By Corollary 1 and Theorem 4 we conclude that there exists a sequence of critical values cm → ∞ as m→ ∞. Theorem 3 is proved. 1. Caffarelli L., Kohn R., Nirenberg L. First order interpolation inequality with weights // Compos. math. – 1984. – 53. – P. 259 – 275. 2. Xuan B. The solvability of quasilinear Brezis – Nirenberg-type problems with singular weights // Nonlinear Anal. – 2005. – 62. – P. 703 – 725. 3. Secchi S., Smets D., Willem M. Remarks on a Hardy – Sobolev inequality // Comptes Rend. Math. – 2003. – 336. – P. 811 – 815. 4. Kang D. Positive solutions to the weighted critical quasilinear problems // Appl. Math. and Comput. – 2009. – 213, № 2. – P. 432 – 439. 5. Alves C., Filho D., Souto M. On systems of elliptic equations involving subcritical or critical Sobolev exponents // Nonlinear Anal. – 2000. – 42. – P. 771 – 787. 6. Deng Z., Huang Y. Existence and multiplicity of symmetric solutions for semilinear elliptic equations with singular potentials and critical Hardy – Sobolev exponents // J. Math. Anal. and Appl. – 2012. – 393. – P. 273 – 284. 7. Ghoussoub N., Robert F. The effect of curvature on the best constant in the Hardy – Sobolev inequality // Geom. Funct. Anal. – 2006. – 16. – P. 897 – 908. 8. Kang D., Peng S. Solutions for semilinear elliptic problems with critical Sobolev – Hardy exponents and Hardy potential // Appl. Math. Lett. – 2005. – 18. – P. 1094 – 1100. 9. Han P. Multiple positive solutions for a critical growth problem with Hardy potential // Proc. Edinburgh Math. Soc. – 2006. – 49. – P. 53 – 69. 10. Cao D., Kang D. Solutions of quasilinear elliptic problems involving a Sobolev exponent and multiple Hardy-type terms // J. Math. Anal. and Appl. – 2007. – 333. – P. 889 – 903. 11. Filippucci R., Pucci P., Robert F. On a p-Laplace equation with multiple critical nonlinearities // J. math. pures et appl. – 2009. – 91. – P. 156 – 177. 12. Xuan B., Wang J. Existence of a nontrivial weak solution to quasilinear elliptic equations with singular weights and multiple critical exponents // Nonlinear Anal. – 2010. – 72. – P. 3649 – 3658. 13. Hsu T. S., Lin H. L. Multiplicity of positive solutions for weighted quasilinear elliptic equations involv- ing critical Hardy – Sobolev exponents and concave-convex nonlinearities // Abstract Appl. Anal. – 2012 / doi:10.1155/2012/579481. 14. Rabinowitz H. Methods in critical point theory with applications to differential equations. – Amer. Math. Soc., CBMS, 1986. 15. Lions P. L. The concentration-compactness principle in the calculus of variations: the limit case. Pt 1 // Rev. Mat. Iberoam. – 1985. – 1. – P. 145 – 201. 16. Mawhin J., Willem M. Critical point theory and Hamiltonian systems // Appl. Math. Sci. – 1989. – 74. 17. Deng Y., Deng L. On symmetric solutions of a singular elliptic equation with critical Sobolev – Hardy exponent // J. Math. Anal. and Appl. – 2007. – 329. – P. 603 – 616. Received 15.12.12, after revision — 18.02.15 ISSN 1027-3190. Укр. мат. журн., 2015, т. 67, № 6